Set Digits Direct is a Sliding Option
There are a lot of options in FitalyStamp. By tapping "i" (for historical reasons - this started as an info menu, but now rather is an options menu) you pop up the FitalyStamp menu. In the "Sliding options..." submenu there is one entry that toggles between "Set digits direct" and "Set punctuation direct". It seems that you are working with the default (again, historically). So tap on "Set digits direct" and you're settled. The occasionally needed punctuation can be reached with sliding.