2024-heraklion-data/notebooks/020_numpy/001_numpy_views_and_copies.ipynb

In [ ]:

import numpy as np

def print_info(a):
    """ Print the content of an array, and its metadata. """
    
    txt = f"""
dtype\t{a.dtype}
ndim\t{a.ndim}
shape\t{a.shape}
strides\t{a.strides}
    """

    print(a)
    print(txt)

NumPy views and copies¶

• Operations that only require changing the metadata always do so, and return a view
• Operations that cannot be executed by changing the metadata create a new memory block, and return a copy

In [ ]:

x = np.arange(12).reshape(3, 4).copy()
print_info(x)

Views¶

Operations that only require changing the metadata always do so, and return a view

In [ ]:

# slice
y = x[0::2, 1::2]
print_info(y)

A view shares the same memory block as the original array.

In [ ]:

z = x.reshape(1, 12)
print_info(z)

CAREFUL: Modifying the view changes the original array and all other views of that array as well!

in place operations¶

In [ ]:

y += 100
print_info(y)

In [ ]:

print_info(x)
print_info(z)

Functions that take an array as an input should avoid modifying it in place!*

Always make a copy or be super extra clear in the docstring.

In [ ]:

def robust_log(x, cte=1e-10):
    """ Returns the log of an array, deals with values that are 0.

    `x` is expected to have non-negative values.
    """
    x[x == 0] += cte
    return np.log(x)
    
# this is not being very clear

In [ ]:

a = np.array([[0.3, 0.01], [0, 1]])

In [ ]:

# This is a view of `a`
b = a[1, :]
print_info(b)

In [ ]:

# what is the output?
robust_log(a)

In [ ]:

# what is the output?
b   # what about b??

Better to make a copy!

In [ ]:

def robust_log(x, cte=1e-10):
    """ Returns the log of an array, deals with values that are 0.

    `x` is expected to have non-negative values.
    """
    x = x.copy()
    x[x == 0] += cte
    return np.log(x)

In [ ]:

a = np.array([[0.3, 0.01], [0, 1]])
b = a[1, :]

#robust_sqrt(a)

In [ ]:

a  # what is the output?   
# b

Copies¶

• Operations that cannot be executed by changing the metadata create a new memory block, and return a copy

• How to find out view or copy?

Choosing row, columns, or individual elements of an array by giving explicitly their indices (a.k.a "fancy indexing") it's an operation that in general cannot be executed by changing the metadata alone.

Therefore, fancy indexing always returns a copy.

In [ ]:

x = np.arange(12).reshape(3, 4).copy()
print_info(x)

In [ ]:

#print(x)
z = x[[0, 0, 2], [1, 0, 3]]
# Can you guess what's z equal to?

print_info(z)

In [ ]:

z += 1000
print_info(z)

# the original array is unchanged => not a view!
print_info(x)

Views are created, when you use other strides to read your data. Slicing and regular indexing allows that, as you know how many byte steps you need to take to get the data.

Fancy indexing does not allow that, because the data you are asking cannot be obtained by just changing the strides. Thus, numpy needs to create a copy of it in memory.