There's no such thing as "over-protective" when it comes to your baby's skin. Since baby's skin is more sensitive than adult skin, it is more likely to burn.
Although complete sun avoidance is sometimes unrealistic, try to limit your baby's exposure as much as possible. Creating preventative habits early on will instill in them the right way to protect themselves against the sun.
Babies under six months of age should not be exposed to the sun. For babies over six months, a sunscreen with an SPF (Sun Protection Factor) of at least 15 is recommended. * SPF is a number on sunscreen labels that indicates how long skin can be in the sun and maintain a low risk for sunburn. The higher the SPF number, the longer it protects a person from burning rays. It's also important that the sunblock you choose have broad spectrum protection against UVB and UVA rays, as both are dangerous. UVB rays cause sunburn, but UVA rays pass deeper into the skin and can cause premature aging. In the long run, both contribute to skin damage, including skin cancer.
Sunscreen should be applied 30 minutes before you leave the house whether you are going to the beach on a sunny day or not. Sitting in the shade doesn't necessarily protect you from the sun either. Remember it's not the heat or visible light that causes sunburn, but rather the invisible UV radiation.
The sun's rays are the strongest between 10 am and 4 pm; so ideally keep your baby out of direct sunlight especially during those hours.
* A doctor should be consulted before using sunscreen on children under 6 months.