|
In my application I wanted to copy 3D array data from linear memory (allocated with cudaMalloc) to a cudaArray with cudaMemcpy3D. Now, I had the problem that cudaMemcpy3D reported cudaErrorInvalidValue. If you encounter a problem like that it might be because of the fact that cudaMalloc was used instead of cudaMalloc3D for allocatimg the linear memory. If cudaMalloc was used then you are forced to create you own cudaPitchedPtr with make_cudaPitchedPtr in order to call cudaMemcpy3D. According to the manual (this information is not extremely emphasized in it) make_cudaPitchedPtris not guaranteed to construct a pitched pointer which is valid for memory copy procedures. One should always use cudaMalloc3D or cudaMallocPitched whenever we want to memcpy because these methods guarantee valid pitched pointers.
“For allocations of 2D and 3D objects, it is highly recommended that programmers perform allocations using cudaMalloc3D() or cudaMallocPitch(). Due to alignment restrictions in the hardware, this is especially true if the application will be performing memory copies involving 2D or 3D objects (whether linear memory or CUDA arrays).” [from the Reference Manual]
This, in fact, solved my problem of the cudaErrorInvalidValue when I tried to cudaMemcpy3D. However, the fact that we use pitched pointer complicates the array access in the kernels. First of all we have to keep track of the data type stored in a pitched pointer (ptr is a void pointer). Furthermore, we have to be careful when dealing with pointer arithmectics because the pitch is, obviously, a byte offset (width of the array). Here is some sample code that shows how to access pitched pointer data. We assume we want to store float4 data in our 3D array which is of size (width, height, depth). In the kernel we want to access the element (x,y,z).
Host Code:
...
cudaPitchedPtr data;
extent = make_cudaExtent(width* sizeof(float4), height, depth);
cudaMalloc3D(&(d_data), extent);
...
Kernel Code:
__global__ kernel(char* data, size_t pitch, ...) {
...
float4 element = *((float4*) (data + (x*sizeof(float4) + y*pitch + z*pitch*height)));
...
}
The one thing I’d like to point out is the fact that we have to use x*sizeof(float4) for accessing the data. This makes sense because data is a char/byte pointer at the time of the pointer arithmetic. Thus, we have to move the pointer by x*sizeof(float4). This musn’t be done in the y and z direction because the sizeof(float4) is already considered in the pitch (the pitch is the width of the array in byte, probably padded for faster memory access).
| 
