yes silicone will build up but over time...I do not like using silicone products because usually(not all the time) they do not spread evenly, and do cause more work for the body shop should you need to paint. Also read labels for "do nots" final inspection says right at the begging of their instructions to use it on a cool ,clean surface. If it's already clean what would be the purpose of using the product. I only make and have used products that can be used on dirty vehicles,
NOT CAKED OR MUDDY must rinse those off, and can be used in the sun. That is where most vehicles are(outside), especially at car shows.