The accidental language designer

When I studied Computer Science at Uppsala University I was required to take a course in Program and Machine Semantics. A very theoretical course and during a particularly difficult lecture our professor tried to motivate us by saying: "In your professional lives you will all design programming languages. To make them successful you'll need to understand the content of this course."
I thought he was nuts. Language design was the domain of geniuses and committees. I didn't feel like a genius and I had no desire to be on a standardization committe, so what did this have to do with me?
On my first job, I began to understand what he meant. The project was to build a test engine for a communication protocol. Tests were to be specified in a simple language that test engineers without a programming background could use, and guess what? We had to come up with the language. We threw together something that seemed reasonable, the lectures on formal semantics seemed far away. Our language turned out to be difficult to use, in situations we hadn't anticipated noone knew what would happen. It didn't have any defined semantics. The system testers eventually learned what worked and what didn't, and used working code as templates for new code. For this problem I think a small language probably was a good idea, but we should have been more careful specifying it.

At another company, a colleague was given a very open-ended optimization problem. He was a great fan of Lisp (so am I, by the way), but company policy mandated C++ as the development language. My colleague despised C++ and object-orientation, he thought it was to "strict". He solved his dilemma by implementing his own "Lisp" in C++ (and then I'm insulting Lisp): everything was represented as arrays of arrays of chars, and all functions worked on arrays of arrays of chars. And, of course, you could also represent code the same way and there was an eval function to run code.
Very flexible. Way too flexible actually, my colleague was the only one who could understand the resulting system. Designing a language is not always the right idea.

Another example, from a third company: a system needs to handle workflows. The workflows are to be specified by domain experts. Now wiser from experience, I suggest that we need to define a language to express these workflows in and document what the constructs of the language mean. But the culture in this company makes this idea impossible. The idea that you can design your own language is just too far-fetched, somewhat like my own reaction many years ago. "The domain experts are supposed to do data entry, not programming!". With time, it turned out the data got more and more complex, and in the end became a sort of language anyway, albeit an odd and undefined one.
Defining a language from the beginning would have been a great idea in this cse, but the organization wasn't ready for it.

So, Professor Barklund, wherever you are these days (Microsoft last I heard): you were right. It's no accident that I have come across language design issues in many of the systems I've worked on: software engineers design programming languages all the time and it's important to understand the semantics of programming languages.
Even more important is to understand when to design a language and when not to. That's harder to teach in a university course, that kind of judgement requires good taste. While an education can be a great help, experience in building systems is what really develops your taste. Design decisions aren't all technical either, the culture you work in is also important. A brilliant solution (in your own not so humble opinion) that your co-workers won't be comfortable with is not a good solution.