It originated from an era predating family names, and meant "from" (also used was zu). This usage is similar with older English ways to call someone, like Robin of Locksley, but carried the specific meaning that the guy was noble as well. Later, the "von" + place name became family names when that became a thing. That being said, it is not totally reliable. Not all nobles head it. And after WWI, when the German peerage was abolished, people could add it to their names with no consequence. All this at least, this is true of the Prussians, because before 1871, there was no "Germany", and I don't know enough to know if you could generalize this to all the constituent kingdoms/principalities/dukedoms/etc that became Imperial Germany.