I have defined 5 temp-tables with multiple indexes (Necessary). Each time I execute a program; a proper cleanup is happening before and after the completion of the program. On running the program, I am getting system memory violation error message and come to know that defined 5 temp-tables structure is not enough to store in entries given in -Bt start up parameter.
Once I increased the entries in -Bt, then memory violation error is not coming. I spent almost two days to find out this issue.
Protrace, db log file and 4GL Trace not helping to find out where exactly the issue is happening. If anyone aware on how to monitor memory violation for such issue, Kindly help me on this
Monitoring memory usage is complex and a lot depends on your platform and the specific version of OpenEdge. Which you have not shared.
It also isn't very likely to be your real problem. This error:
SYSTEM ERROR: Memory violation. (49)
Unless you are messing with MEMPTRs this is always an OpenEdge bug. It doesn't have anything to do with "having enough (memory) to store". It means that somewhere in the 4GL interpreter an illegal memory reference was made as described here: https://community.progress.com/s/article/3790 It is also very frequently a "Heisenbug". Protrace will tell you that the error occurs on perfectly innocent lines of code. It can be very frustrating but it doesn't mean that you didn't have enough memory.
Increasing -Bt increases the memory available to buffer temp tables but that memory is for performance purposes, it is not a limit on temp-table size. If your temp-tables exceed that size they are paged to disk and show up as growing DBI* files in the session temporary directory (the -T startup parameter).
Increasing -Bt may have "cured" your problem indirectly by moving things around so that the error will now occur under different circumstances - perhaps so different that you don't see it happen again. Or it happens much less frequently. Or it only happens in Production instead of Test... I had one of these that haunted me for years. It kept moving around and coming and going randomly. I'd think I had it cured, sometimes for months at a time, but it would always eventually pop up again. Ultimately it turned out to be a bug in the internal (to Progress) implementation of a certain rarely used 4gl feature.