Working with EVP and OpenSSL, coding in C

I've seen many questions on OpenSSL and EVP, but not very many clear answers, but I figured I'd still post my question here and hope for better feedback.

The materials given to me are a signed file "symmetrickey.bin", an RSA key set "privatekey_A.pem", "publickey_A.pem", and the other user's public key "publickey_B.pem".

What I need to do is:

1. Unsign symmetrickey.bin and store it into a text file.
2. Encrypt a message.txt using the symmetrickey.txt and some algorithm like AES for example.
3. Sign the encrypted message with privatekey_A.pem and write to a file cipher.bin.
4. After that I need to unsign and verify the signature on cipher.bin.
5. Then decrypt the message with our symmetric key then write to another file.

The issues I'm having are understanding how to implement the OpenSSL EVP libraries. The API page is not very clear about where the values for each function comes from. For example, EVP_OpenInit() where am I getting ek or the length of ek "ekl"? Is "prvi" a private key? And how am I to know the type? These are things I'm not given.

I've looked at many implementations and most don't answer my questions or they give crazy code with little to no explanation of what's going on or where values are coming from. I'm posting here as a last resort...

For the sign/unsign key part I need further information, how is this signature done? For example, is this signature an X byte length at the end of the file and can then easily be removed?

For items 2-5 in your list the following code will surely assist, it is based on the examples from openssl documentation with more comments and adaptations for your needs. Feel free to ask if you have any questions that are not commented!

crpytor.c

#include <string.h>
#include <stdio.h>
#include <unistd.h>
#include <openssl/evp.h>

#define APPNAME "C"

#define CHUNK_SIZE 512
int do_crypt(FILE *in, FILE *out, int do_encrypt)
{
    /* Allow enough space in output buffer for additional block */
    unsigned char inbuf[CHUNK_SIZE];
    unsigned char outbuf[CHUNK_SIZE + EVP_MAX_BLOCK_LENGTH];
    int inlen;
    int outlen;
    EVP_CIPHER_CTX ctx;
    /* Bogus key and IV: we'd normally set these from
     * another source.
     */
    unsigned char key[] = { 0x13, 0xa3, 0xb4, 0xc1, 0x24, 0x19, 0xf5, 0x23, 0x18, 0xef, 0xca, 0x12, 0x4c, 0x9f, 0x14, 0xfe };
    unsigned char iv[] = { 0x92, 0x1c, 0x23, 0x3f, 0x5e, 0x10, 0x3d, 0x9a };
    /* Don't set key or IV because we will modify the parameters */
    EVP_CIPHER_CTX_init(&ctx);
    /* Using Blowfish encryption with cbc algorithm, you can use whichever is supported in openssl if you wish */
    EVP_CipherInit_ex(&ctx, EVP_bf_cbc(), NULL, NULL, NULL, do_encrypt);
    EVP_CIPHER_CTX_set_key_length(&ctx, 16);
    /* We finished modifying parameters so now we can set key and IV */
    EVP_CipherInit_ex(&ctx, NULL, NULL, key, iv, do_encrypt);
    for(;;)
    {
        inlen = fread(inbuf, 1, CHUNK_SIZE, in);
        if(inlen <= 0) break;
        if(!EVP_CipherUpdate(&ctx, outbuf, &outlen, inbuf, inlen))
        {
            /* Error */
            EVP_CIPHER_CTX_cleanup(&ctx);
            return -1;
        }
        fwrite(outbuf, 1, outlen, out);
    }
    if(!EVP_CipherFinal_ex(&ctx, outbuf, &outlen))
    {
        /* Error */
        EVP_CIPHER_CTX_cleanup(&ctx);
        return -1;
    }
    fwrite(outbuf, 1, outlen, out);
    EVP_CIPHER_CTX_cleanup(&ctx);
    rewind(in);
    rewind(out);
    return 0;
}

/* This is the standalone encryptor entry point */
int main(int argc, char** argv)
{
    FILE *encode_file;
    FILE *decode_file;
    int enc_or_dec;
    if (argc < 4)
    {
        printf("Usage: %s [plain file] [encrypted file] [0/1 deccrypt/encrypt]\n", argv[0]);
        return -1;
    }
    encode_file = fopen(argv[1], "r");
    decode_file = fopen(argv[2], "w+");
    /* Stupid decimal translation */
    enc_or_dec = *argv[3]-48;

    do_crypt(encode_file, decode_file, enc_or_dec);
    return 0;
}

And the Makefile:

all:
    gcc cryptor.c -o cryptor -g -lcrypto -I ../openssl-1.0.1f-host/include
clean:
    rm cryptor

This code does not use EVP_OpenInit() because it is used only for decryption, while my method (and your needs) require both encryption or decryption. While you can use EVP_OpenInit() to initialize a decryption context, I replaced the single call suitable only for decryption with two calls suitable for both encryption and decryption.

From the man page:

EVP_OpenInit() initializes a cipher context ctx for decryption with cipher type. It decrypts the encrypted symmetric key of length ekl bytes passed in the ek parameter using the private key priv. The IV is supplied in the iv parameter. EVP_OpenUpdate() and EVP_OpenFinal() have exactly the same properties as the EVP_DecryptUpdate() and EVP_DecryptFinal() routines, as documented on the EVP_EncryptInit(3) manual page.

EVP_OpenInit() for key files

If the signed file you are referring to is a public key file in RSA/DSA or similar format, you can use this StackOverflow question for a better method than mine as it does the key extraction from the file automatically (and uses EVP_OpenInit() like you require)