In a medium bowl, whisk together flour, baking soda, and cream of tartar.
In a large bowl, mix butter, powdered sugar, eggs, vanilla, and lemon extract until combined.
Add the flour mixture to the wet mixture and blend in 2-3 batches, beating just until fully combined.
Divide dough and put it into plastic wrap packets; chill 2-3 hours.
Preheat oven to 375 degrees.
Roll dough to 1/4″ thickness and cut with cookie cutters.
Place on parchment paper on baking sheet and bake 6 minutes. Cookies are done when their centers are puffy. Do not overbake.
Cool cookies.
Make Frosting
Beat together powdered sugar, vanilla, and butter.
Mix until the frosting comes together, then add milk a little bit at a time until it is the consistency you want. You will probably not add all the milk.
If you want to make multiple colors of frosting, separate the frosting into multiple containers at this point.
Add a few drops of food coloring until it is your desired color.