Combine milk base and sweetener of choice, pumpkin puree and pumpkin pie spice in a small saucepan over medium heat. Stir continuously until sugar is well dissolved.
Reduce heat to low and add extract. Allow to steam another 3-4 minutes.
Remove from heat and strain pumpkin puree out by using a fine mesh strainer over a glass jar.
Allow to cool slightly. Store in a jar or bottle with a lid in the fridge for 5-6 days.