I have an algorithm for steganography on h264 (AVC). But I am unable to find a library or equations for H264 compression. The steps are DCT-> Quantization-> Entropy encode**. Can anyone provide me the step by step compression code or equations for these steps? The intention is to manipulate the LSBs after the entropy encoding step. I am using EmguCV and C# for my project. I am also using FFMpeg. I have a very little knowledge on C and C++.
x264 is the best library for h.264 encoding. Its also open source. As for the 'step by step equations' you need to read the ISO/IEC 14496-10 document. Its about 730 pages.