In a bowl, mix together the coconut milk, almond milk, erythritol sweetener, and vanilla.

Pour the mixture into 3-ounce ice cream molds and freeze for 90 minutes. Insert the wooden popsicle sticks and freeze again for at least 7 hours. If you have molds with sticks, insert the sticks and add the mixture. Freeze for at least 8 hours.

In the top of a double boiler, melt the chocolate and coconut oil, stirring. Remove from the heat and let the chocolate mixture sit to return to room temperature.

Line a baking sheet with parchment paper. Working with 1 ice cream bar at a time, remove the ice cream bar from the mold and dip into the melted chocolate. Use a spoon to coat the bar evenly, if needed. Once dipped, arrange the bar on the prepared baking sheet and immediately place it in the freezer. Repeat for the remaining bars.

Freeze for at least 1 hour.