That's right: there are no flowers, no bushes, no ornamental grasses of any kind, only peppers...and you're guess as to exactly why this garden exists is as good as mine. The marble statue at its center is inscribed: Moriz Ritter von Franck, schöpfer des stadtparkes (creator of the city parks), and has been heavily vandalized with what looks like black sharpie.
As I stood at the feet of von Franck marveling at the peculiar bed of capsaicin-laced greenery in which he was standing, I wondered, "did he have a particular taste for spicy food?" Or, I thought with some embarrassment, "maybe he was a terrible person that no one liked and they thought the peppers planted at his feet might be a more civilized way of saying they hope he's burning in Hades." Of course if that were the case, why would they go to all the trouble of building the statue in the first place? Maybe he was an energetic guy with a fiery temper. Or maybe he just had a quirky sense of humor...the sort best memorialized in just such an oxymoronic manner.
When I got home I looked him up on Wikipedia and (with lots of help from google translate, as there was no version of the article in English) learned that he had been the Bürgermeister of Graz from 1861-1864 and again from 1867-1870. During his tenure he instigated a house numbering system, improved city plumbing, created public schools and a state hospital, and pursued the beautification of Graz by designing a system of city parks. Sounds like a decent enough guy--it made me a little sad to think of his likeness being scribbled over with black marker.
In any case the mystery of the ripening peppers remains. There must be some odd bit of local lore that explains their existence within this town of chips-n'-ketchup eating menschen, and I'll be sure to let you know if I hear of it. In the meantime, I wonder how much trouble I'd get in if I picked one or two to liven up my uneaten jar of ketchup salsa...