Ability to transfer your residence permit to another EU country

It was really a surprise to me when I moved to an EU country that every state functions completely unsynced in terms of residence permits and citizenship statuses.

If I get a residence permit for work in Germany and then decide to move to Austria or Poland, I should be able to transfer the "years I've lived in the country" across all EU nations.

Just imagine how strange that sounds. A person who lived in New York state and moved to California state would need to start everything from scratch.