Pea Gravel Coming to Surface when Troweling

No big deal as I'm only working on a slab for a workshop, but was curious. I have to work some of the concrete with a trowel as I poured around some 6 X 6 posts. As I'm working some of the concrete around the posts, pea gravel is working its way up to the surface (doesn't make for a very smooth surface). Perhaps it's all a matter of practice. Thank you