I'm trying to read the data which is not structured well. It looks something like this
Generated by trjconv : P/L=1/400 t= 0.00000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 18.53 -9.69 4.68
4P2 aP2 4 55.39 74.34 4.60
5P3 aP3 5 22.11 68.71 3.85
6P3 aP3 6 -4.13 24.04 3.73
7P4 aP4 7 40.16 6.39 4.73
8P4 aP4 8 -5.40 35.73 4.85
9P5 aP5 9 36.67 22.45 4.08
10P5 aP5 10 -3.68 -10.66 4.18
Generated by trjconv : P/L=1/400 t= 1000.000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 18.53 -9.69 4.68
4P2 aP2 4 55.39 74.34 4.60
5P3 aP3 5 22.11 68.71 3.85
6P3 aP3 6 -4.13 24.04 3.73
7P4 aP4 7 40.16 6.39 4.73
8P4 aP4 8 -5.40 35.73 4.85
9P5 aP5 9 36.67 22.45 4.08
10P5 aP5 10 -3.68 -10.66 4.18
Generated by trjconv : P/L=1/400 t= 2000.000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 18.53 -9.69 4.68
4P2 aP2 4 55.39 74.34 4.60
5P3 aP3 5 22.11 68.71 3.85
6P3 aP3 6 -4.13 24.04 3.73
7P4 aP4 7 40.16 6.39 4.73
8P4 aP4 8 -5.40 35.73 4.85
9P5 aP5 9 36.67 22.45 4.08
10P5 aP5 10 -3.68 -10.66 4.18
Generated by trjconv : P/L=1/400 t= 3000.000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 18.53 -9.69 4.68
4P2 aP2 4 55.39 74.34 4.60
5P3 aP3 5 22.11 68.71 3.85
6P3 aP3 6 -4.13 24.04 3.73
7P4 aP4 7 40.16 6.39 4.73
8P4 aP4 8 -5.40 35.73 4.85
9P5 aP5 9 36.67 22.45 4.08
10P5 aP5 10 -3.68 -10.66 4.18
It consists of different frames with updated time. What I showed here is just a sample. The whole file is around 50GB. therefore it will be better to read it line by line or in chunks. But I could not figure out how to deal with the headers of each frame. Are there any ways to get rid of these headers? For now I used following method:
import numpy as np
#define a np.dtype for gro array/dataset (hard-coded for now)
gro_dt = np.dtype([('col1', 'S4'), ('col2', 'S4'), ('col3', int),
('col4', float), ('col5', float), ('col6', float)])
file = np.genfromtxt('sample.gro', skip_header = 2, dtype=gro_dt)
But it throws the following error when it comes to next header.
ValueError: Some errors were detected !
Line #13 (got 7 columns instead of 6)
Line #14 (got 1 columns instead of 6)
Line #25 (got 7 columns instead of 6)
Line #26 (got 1 columns instead of 6)
Line #37 (got 7 columns instead of 6)
Line #38 (got 1 columns instead of 6)
Write an adaptor that strips the periodic headers.
def adapt(f):
for line in f:
if line.startswith("Generated"):
print(line, end='')
# Consume the following line as well.
# If your data is well behaved, you can
# assume the following line exists and should be
# skipped, instead of using the try statement.
try:
print(next(f), end='')
except StopIteration:
pass
continue
yield line
with open('sample.gro') as f:
file = np.genfromtxt(adapt(f), dtype=gro_dt)