Way too much resources are used with Pytorch
I am using pytorch to train a DQN model. With ubuntu, if I use htop, I get
As you can see, all resources are used and I am a bit worried about that. Here is my code.
Is there a way to use less resource? Do I have to add my requirement using pytorch?
Be aware that there's no GPUs on my machine, just CPUs
Yes, there is. You can use torch.set_num_threads(...) to specify the number of threads. Depending on the PyTorch version you use, maybe this function will not work correctly. See why in this issue. In there, you'll see that if needed you can use environment variables to limit OpenMP or MKL threads usage via OMP_NUM_THREADS=? and MKL_NUM_THREADS=? respectively, where ? is the number of threads.
Keep in mind that these things are expected to run on GPUs with thousands of cores, so I would limit CPU usage only when extremely necessary.
- Is there a good way to distinguish spurious wake up and Thread.interrupt()?
- std::lock_guard or std::scoped_lock?
- Synchronizing on String objects in Java
- Point Cloud Library (PCL) filtering timeout
- Managing multiple exclusive locks for a set of unique values
- Run external command in bg (other thread), capture output when available for rendering in TUI
- Do Java 21 virtual threads address the main reason to switch to reactive single-thread frameworks?
- Why did entering a lock on a UI thread trigger an OnPaint event?
- What's the difference between "static" and "dynamic" schedule in OpenMP?
- What is the best way to make a parent thread wait after initializing a child pthread until it receives some signal from the child?
- How to implement background polling in WinUI application in c#
- Java multithreading concept and join() method
- Detaching a thread versus calling pthread_exit() from main(), consequences for resources and memory in both scenarios
- Multiprocessing example giving AttributeError in Jupyter Notebook
- Peterson's Algorithm's behavior for various optimization flags
- ForkJoinPool.commonPool's threads are created new indefinitely instead of reused/closed when calling CompletableFuture methods of OpenFGA
- Is a function pointer variable always defined/legal for multithreading (thread-safe)(atomic) in C?
- Using multi-cores for increased parallel performance
- How to use a managed Tauri state variable inside a spawned tauri async runtime thread?
- Counter value is not decreasing while using multithreaded loop
- how to use handler on android concurrently?
- Using threading to load textures in Raylib is failing
- Problems implementing properly a Parallel.ForΕach loop and continue other work in the mean time
- Why is "kill" is not exiting a thread immediately?
- Synchronization of non-final field
- Can Virtual Threads improve querying a database in Java?
- Running code in main thread from another thread
- Understanding the volatile Modifier in the Context of x86 Architecture and the Java Memory Model (JMM)
- Is assigning a pointer in C program considered atomic on x86-64
- Removing items from bottom of ConcurrentStack<T> or equivalent functionality