The Bible tells us (Colossians 2:11-15) that in the New Covenant, baptism replaces circumcision in the Old. In the Old Testament God required circumcision as the basic way of bringing a child into the Covenant (Genesis 17:1-14). The ritual was symbolic of the stripping away of hardheartedness and obstinacy so that the person could enter into God's covenant of love with his people (Deuteronomy 10:12-22; 30:1-10). It was through circumcision that the child was brought into the Covenant by his parents, as Jesus was, long before he knew what was happening.
We bring children to Christ first through the waters of baptism. In Matthew's Gospel, the Greek word used for children is paidía, a general word for children that does mean children of any particular age. In Luke's Gospel, however, we another word is used in addition: bréphe, which means infants, children who are too young to walk. These infants, Luke tells us, were being brought to Christ by adults so that Christ "might touch them". In the sacrament of baptism, Christ touches the child with his grace and life.
So Christian parents bring their child into the covenant with God by baptism.
But - you might ask - does this mean anything if it is not the faith of the person that brings him or her into the Covenant?
The Bible gives us many examples of the faith of one person affecting the salvation of another. For example:
Abraham intercedes with God on behalf of the city of Sodom. (Gen 18:16-33)
The centurion's servant is healed, not because of his own faith, but because of the centurion's faith in Christ. (Mt 8:5-13; cf. Lk 7:1-20)
The Canaanite woman's daughter is healed because of her mother's appeal to Christ. (Mt 15:21-28)
The cripple's friends bring him to Christ because he cannot walk himself. Jesus saw their faith (not the cripple's), and said to the crippled man, "Your sins are forgiven". So Christ forgives sins (as he does in baptism) through the faith of a third party - the child's parents, for example.
In Acts 16:27-33 St. Paul answers the gaoler's question, "What must I do to be saved?" by telling him that he must believe. We know that St. Paul points out that faith in Christ and repentance of one's sins is necessary to receive the sacrament of baptism. However, we see that not only the man was baptised, but also "the rest of his household", which probably included children.
Other passages that speak about "whole families" or "entire households" being baptised (which almost certainly included children) are Acts 16:15 and 1 Cor 1:16.
Take no notice of "born-agains" who keep going on about your baptism being no good. If they want to put aside over 1,500 years of Christian teaching since the very earliest Christian times, that is their concern. Don't let them make you think you have a problem!