I don't think there is a right answer or a wrong one for your question. I believe that every person has a way of dealing with been alone and been around other people. For your description you have a lot of people around and that mean that for "survival purposes" you are covered.
But I often think about the way we believe that this or that relationship must be, according to the social codes and rules about what is "normal".
I think we grow up seen beautiful movies reading books and consuming a lot of narratives that says "have a family, have a good friend to drink with on fridays, have a network that make you seem like a nice person to be around". That is a perception of life that not everyone can or want or need to follow.
If you don't think you need friends but yet this is something that make you feel like your doing something wrong, maybe you could take a look at how you build this notion of "right" in your head. Maybe the "right" don't apply in your affections network
And it is perfectly fine.

I believe that you need to have people you trust around you. But regarding only your relationship with your wife, maybe it is a good thing be discussed between you two. Is she worry that you don't have friends? Does she has her friends? Is there someone you can reach if one day you have an issue with your wife and you need support?
Those are all very hard questions to answer and they are related to a lot of really personal feelings and beliefs that I belongs only to you.
But I think they should be on your sight.

I hope I could help and if you ever need to talk about it, feel free to write.
I don't have a lot of friends around too, but I never lose perspective that I need people around, even if is online and not often. Just in case I need to talk. And I also have therapy sessions once a week, they really help me because is a place that I can say whatever o want. Something like a diary, but it talks back to you.

Best,