I'm about to rewrite simple application where speed is very important.
Which approach is faster: using the pure PHP in the HTML files or using a template engines like Smarty or Twig?
Moreover, which is parsed faster? Is the Smarty cache for example faster than using pure PHP?
Which of the template engines is the fastest?
"Depends" is the answer to all your questions.
What is "faster"? Execution time? Development time? Maintenance? Memory overhead? A mixture of them? A template engine is usually trading in some performance (speed, memory) for better development and maintenance.
If you are talking about purely dynamic templating (meaning: template evaluated on every request) PHP will outrun any template engine. This is a nobrainer, really. If you're taking caching into account, a template engine like Smarty may help. Caching is nothing you couldn't implement yourself in plain PHP, though. With Smarty it's just been done for you (and on a far more sophisticated level than you possibly would).
If you are using a framework, say Symfony, it might be wise to use Twig, as Twig and Symfony are tightly integrated. Sure you can use Smarty or plain PHP. The question here is: is it practicable?
Caching makes sense when building sites from datasources like a database or remote APIs. What you are really saving (in a sense of reducing) here are database calls, intensive calculations, etc. Check if you have any time-intensive functions running to build your site. If so, use caching (if you can).
Knowing development/maintenance/convenience/performance trade-offs, I would (always) recommend using a template engine. Being a Smarty developer, I'll, of course, suggest using Smarty. That is unless you're using Symfony, then you might be better of with Twig. Or some other framework featuring some other template engine.
Please ignore posts like Smarty vs. Twig, as they only compare a very limited view of the engines. Don't trust benchmarks you haven't faked yourself™.
In general, though, Smarty 3.1 is a bit faster than Twig. Twig is doing a lot of stuff at runtime (being the time when a template is executed) that Smarty does on compile time (being the time when a template is prepared for execution). Twig is not really pissing away speed here. Twig needs to do certain stuff at runtime by design. They traded a bit of performance for a bit of "convenience" (Accessing arrays and objects with the same notation, for example).