As far as I've learned, I have to keep the private key secure and publish the public key, so that anyone can encrypt any data using public key and I can decrypt it using my own private key (which nobody has).
The question is, what if I publish the private key and keep the public key? Again the algorithm seems to work: anybody locks the data with the private key (which is published), but no one has the public key (which is kept secure by me).
What makes a public key, a public key? What secure and important data is stored on private key which I should show no one?
For a complete encrypt + decrypt (or sign + verify) process you always need BOTH keys. One for encryption/signing and the other one for decryption/verifying. Which of the keys you use for which operation is (in principle) irrelevant, as long as you use the respective other one for the inverse operation.
So let's assume you used a tool like openssl to generate a key-pair A and B.
When it comes to publishing one of those keys, we have to take into account two aspects
Math: From a pure mathematics point of view (and leaving out the security for a moment), it's irrelevant which of the keys you make public and which you keep private. All processes will work either way.
decrypt(encrypt(data, A), B) == data == decrypt(encrypt(data, B), A)
verify(sign(data, A), B) == OK == verify(sign(data, B), A)
Security: When we take security into account, proving your identity via digital signature is only possible, if you use a key, nobody else can possibly know. For certain crypto systems, it's possible to derive key A from key B, ie there exists a function such that
A = f(B)
but not vice versa, ie there is no function such that
B = f(A)
Thus, the moment you know B, you also know A, but if you only know A, there is no possibility you can derive B.
Thus B is called the private key and must kept private, A is the public key, which can be published. If you do it the other way around, the processes will still work from a mathematical point of view (although most systems will reject your keys), but they are not secure anymore ...