That’s correct. If you make a change to the local database, the same changes won’t be reflected in production. That’s generally a good thing, because you can break things, drop tables, and reset schema in testing without it breaking your production database.
Heroku has built-in ways to allow for database uploads, if you need to migrate the rows themselves. However, if you need to do that, you’ll want to make sure you’re using Postgres both locally and in production (not SQLite like in this tutorial). Beware, setting up Postgres locally is notoriously a pain in the neck, but once you’ve done it, you can use it for all your apps.
Obviously Django takes care of all model changes. If you’re just updating models, this setup (SQLite local, Postgres production) works just fine!