I'm looking for general advice for dealing with large, highly sparse inputs/outputs within OpenMDAO.
I have recently implemented a new group within my existing OpenMDAO model. All analytic partials have been declared along with Jacobian sparsity using the row/col method. Several highly sparse (2D and 3D) matrices are currently passed through various components. These matrices can significantly increase in size ([10000+, 10000+] would not be uncommon) depending on the application of the group.
As I expected, given the addition of this new group, my overall optimization time has increased. However, this increase in runtime is far too large for my model's application.
I am also worried about hitting memory allocation caps due to the potential size of these inputs and outputs (I had already hit these caps before I implemented Jacobian sparsity).
I have been looking into utilizing scipy's sparse library within each ExplicitComponent's compute() and compute_partials(), which has slightly reduced optimization runtimes thus far.
I do not believe ExplicitComponents accept sparse matrix inputs/outputs (e.g. scipy.sparse.coo_matrix). So, I have been exploring only passing in the non-zero data within my sparse matrices, reassembling the sparse matrix within the compute, then extracting the non-zero output data. However, this process is fairly tedious, and the reassembly process seems like a redundant computation. Moreover, I can't always guarantee I know the sparsity pattern of each matrix. I had hoped decreasing the size of the inputs/outputs in this way may help reduce the overhead of passing around these large matrices between subsystems and help to generally reduce the gradient evaluation time.
Ultimately, I am looking for any advice specific to the OpenMDAO framework regarding the handling of large sparse matrices, specifically regarding runtime reduction and memory allocation. Does OpenMDAO include any support for sparse inputs/outputs in a similar fashion to declaring the sparsity of Jacobians?
The way I've handled this has been to take a similar approach to you, and store a scipy sparse, or pydata sparse matrix as an attribute to the component, and then assign the underlying data values which would come from the inputs.
self._M.data[:] = inputs['M_nz']
When you say that you don't know your sparsity pattern, are you saying that it changes from call to call?