Pytorch中的MapTensor

wpcxdonn  于 2022-11-23  发布在  其他
关注(0)|答案(3)|浏览(141)

我有以下两个Tensor:

*img是形状(224,224,3)的RGB图像
*uvs是具有相同空间大小的Tensor,例如(224,224,2),其Map到坐标(x,y)。换句话说,其为输入图像的每个像素提供(x,y)坐标。

现在我想创建一个新的输出图像Tensor,它包含索引(x,y)上的输入图像的值,所以输出应该也是一个像素根据MapTensor重新排列的图像。
小玩具举例:

img = [[c1,c2], [c3, c4]] where c is a RGB color [r, g, b]
uvs = [[[0,0], [1,1]],[[0,1], [1,0]]]
out = [[c1, c3], [c4, c2]]

如何在pytorch中以快速矢量化的方式实现这样的事情呢?

7jmck4yq

7jmck4yq1#

尝试使用:

out = img[idx[...,0], idx[...,1]]
3gtaxfhh

3gtaxfhh2#

我能够解决它(在Quang Hoang的帮助下回答)

out[idx[...,0], idx[...,1]] = img
ajsxfq5m

ajsxfq5m3#

您需要的是torch.nn.functional.grid_sample()。您可以这样做:

width, height, channels = (224, 224, 3)

# Note that the image is channel-first (CHW format). In this example, I'm using a float image, so the values must be in the range (0, 1).
img = torch.rand((channels, height, width))

# Create the indices of shape (224, 224, 2). Any other size would work too.
col_indices = torch.arange(width, dtype=torch.float32)
row_indices = torch.arange(height, dtype=torch.float32)
uvs = torch.stack(torch.meshgrid([col_indices, row_indices]), dim=-1)

# Transform the indices from pixel coordiantes to the to the range [-1, 1] such that:
#   * top-left corner of the input = (-1, -1)
#   * bottom-right corner of the input = (1, 1)
# This is required for grid_sample() to work properly.
uvs[..., 0] = (uvs[..., 0] / width) * 2 - 1
uvs[..., 1] = (uvs[..., 1] / height)* 2 - 1

# Do the "mapping" operation (this does a bilinear interpolation) using `uvs` coordinates.
# Note that grid_sample() requires a batch dimension, so need to use `unsqueeze()`, then
# get rid of it using squeeze().
mapped = torch.nn.functional.grid_sample(
    img.unsqueeze(0),
    uvs.unsqueeze(0),
    mode='bilinear',
    align_corners=True,
)
# The final image is in HWC format.
result = mapped.squeeze(0).permute(1, 2, 0)
  • 旁注:我发现你的问题是通过搜索一个相关问题的解决方案,我有一段时间。当我写你的问题的答案,我意识到什么bug是导致我所面临的问题。通过帮助你,我有效地帮助了我自己,所以我希望这能帮助你!*:)

相关问题