The "Core" or "Joomla Core" is a technical term for the files and folders that make your bear-bone Joomla after you just installed it.
Joomla is an extendable system. You can add some extra features to your Joomla. Just like you can add extra programmes to your operating system.
If you installed one the of extra features on your website, Joomla core is what your website is without those additional features.