Don't worry!!!
My son is now 6 months old and as a very young baby he would only sleep (during the daytime) in his infant carrier . He slept ok in the cot at night but he would never sleep in it during the daytime; I had to take him for walks in the carrier 3 times a day to get him to sleep! Gradually, when he was 3-4 months old, I managed to get him to fall assleep if I took him for walks in his pushchair, he would cry for only a few minutes and then settle down. Then by 5 months he would sleep for about 30 minutes in his cot if I held him for a few minutes first. Now he sleeps for over an hour in the cot in the daytime! I think babies under 3 months need the security of close human contact, and I don't regret giving my son this, I like the fact that he felt secure and didn't feel abandoned. Don't worry, you'll be able to wean him off the carrier when he's a little older and you're not spoiling him, just giving him love and security.