My theory is that dogs are able to 'read' people, and know who they feel comfortable with. A toddler's mind is busy, with disconnected thoughts, short attention span, and surprise movements, so they are more difficult to 'read'.Generally, only those that are around children a lot have become trusting of them.