Numpy Array2string Just Writing ... In String?

I have a simple thing to do, read some vectors and write them in a file. The vectors are 1024 dimensional. for emb in src: print(len(emb[0].detach().cpu().numpy())) #

Solution 1:

The normal way of writing a 2d array to a text file (so it can be read back) is with np.savetxt:

In [309]: src = np.random.rand(6,4)
In [310]: src
Out[310]: 
array([[0.78756364, 0.11385762, 0.16631052, 0.10987765],
       [0.59954504, 0.80417064, 0.22461205, 0.47827772],
       [0.10993457, 0.11650874, 0.55887911, 0.71854456],
       [0.53572426, 0.55055622, 0.25423811, 0.46038837],
       [0.05418115, 0.50696182, 0.31515915, 0.65310375],
       [0.81168653, 0.81063907, 0.95371101, 0.11875685]])

write:

In [311]: np.savetxt('test.txt', src, fmt='%10.6f',delimiter=',')
In [312]: cat test.txt
  0.787564,  0.113858,  0.166311,  0.109878
  0.599545,  0.804171,  0.224612,  0.478278
  0.109935,  0.116509,  0.558879,  0.718545
  0.535724,  0.550556,  0.254238,  0.460388
  0.054181,  0.506962,  0.315159,  0.653104
  0.811687,  0.810639,  0.953711,  0.118757

Test the loading:

In [314]: np.genfromtxt('test.txt',delimiter=',')
Out[314]: 
array([[0.787564, 0.113858, 0.166311, 0.109878],
       [0.599545, 0.804171, 0.224612, 0.478278],
       [0.109935, 0.116509, 0.558879, 0.718545],
       [0.535724, 0.550556, 0.254238, 0.460388],
       [0.054181, 0.506962, 0.315159, 0.653104],
       [0.811687, 0.810639, 0.953711, 0.118757]])

savetxt does a formatted write, row by row, roughly like:

In [315]: fmt = ','.join(['%10.6f']*4)
In [316]: fmt
Out[316]: '%10.6f,%10.6f,%10.6f,%10.6f'
In [317]: for row in src:
     ...:     print(fmt%tuple(row))    # f.write(...)
     ...:     
  0.787564,  0.113858,  0.166311,  0.109878
  0.599545,  0.804171,  0.224612,  0.478278
  0.109935,  0.116509,  0.558879,  0.718545
  0.535724,  0.550556,  0.254238,  0.460388
  0.054181,  0.506962,  0.315159,  0.653104
  0.811687,  0.810639,  0.953711,  0.118757

In fact I can wrap that in file write:

In [318]: with open('test1.txt','w') as f:
     ...:     for row in src:
     ...:         print(fmt%tuple(row), file=f)
     ...:     
In [319]: cat test1.txt
  0.787564,  0.113858,  0.166311,  0.109878
  0.599545,  0.804171,  0.224612,  0.478278
 ...

Solution 2:

The vectors are still 1024 dimensional, but the display is only showing a curtailed view of the array.

You can view the whole array by setting print options:

import numpy as np
np.set_printoptions(threshold=np.nan)