arrayspytorchdataset

PyTorch: difference between reshape() and view() method


What is the difference between reshape and view method and why do we need and I am using pytorch tensors and working on changing the shape of data then I came to know these two functions. what are the affects on memory which consumes the more memory and which is more expensive if we are working on large data with less resources.

x = torch.tensor([1, 2, 3, 4, 5], dtype = torch.float32)
x = x.reshape(-1, 1)

and the view method

x = x.view(x.shape[0], 1)

What is the difference and which should I use


Solution

  • The short answer: When reshaping a contiguous tensor, both methods will do the same (namely, provide a new view of the given tensor) and can therefore be used interchangeably. When reshaping a non-contiguous tensor, reshape() will duplicate the necessary parts of memory from the given tensor for producing the resulting tensor, while view() will fail with a RuntimeError.

    The long answer

    The main difference is how torch.Tensor.reshape() and torch.Tensor.view() handle non-contiguous tensors.

    To understand the difference, we need to understand what is a contiguous tensor, and what is a view of a tensor:

    Now back to the two methods:

    We can demonstrate this with the following piece of code:

    from torch import arange
    
    contiguous = arange(16).view(4, 4)             # Create contiguous 4×4 tensor
    noncontiguous = arange(20).view(4, 5)[:, :4]   # Create non-contiguous 4×4 tensor
    
    contiguous_r = contiguous.reshape(16)          # OK: produces a 1-d view
    assert contiguous_r.data_ptr() == contiguous.data_ptr()  # Same memory used
    
    contiguous_v = contiguous.view(16)             # OK: produces a 1-d view
    assert contiguous_v.data_ptr() == contiguous.data_ptr()  # Same memory used
    
    noncontiguous_r = noncontiguous.reshape(16)    # OK: produces a new 1-d array
    assert noncontiguous_r.data_ptr() != noncontiguous.data_ptr()  # New memory used
    
    noncontiguous_v = noncontiguous.view(16)       # ERROR: cannot produce view
    

    The last line will produce RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

    Maybe at this point, I should also mention what a tensor's stride is: in essence, it is the information that tells us how to map the tensor's indexes to its underlying memory. You will find more information on strides in particular and on contiguous vs. non-contiguous tensors in general, for example, in this discussion in the PyTorch forum.

    As to your question, which should I use?: