Michael Pfeuti

Yesterday I discovered a great libraray for OpenGL/DirectX which allows you to easily add an interface to change varibales. It is called AntTweakBar.

What needs to be done is just to tell the library what your variables are and of what type they are. The rest is done by the library. So if you are looking for an easy-to-use but still visually appealing way to edit certain variables of your OpenGL/DirectX application this is certainly a library to look into.

In my application I wanted to copy 3D array data from linear memory (allocated with cudaMalloc) to a cudaArray with cudaMemcpy3D. Now, I had the problem that cudaMemcpy3D reported cudaErrorInvalidValue. If you encounter a problem like that it might be because of the fact that cudaMalloc was used instead of cudaMalloc3D for allocatimg the linear memory. If cudaMalloc was used then you are forced to create you own cudaPitchedPtr with make_cudaPitchedPtr in order to call cudaMemcpy3D. According to the manual (this information is not extremely emphasized in it) make_cudaPitchedPtris not guaranteed to construct a pitched pointer which is valid for memory copy procedures. One should always use cudaMalloc3D or cudaMallocPitched whenever we want to memcpy because these methods guarantee valid pitched pointers.

“For allocations of 2D and 3D objects, it is highly recommended that programmers perform allocations using cudaMalloc3D() or cudaMallocPitch(). Due to alignment restrictions in the hardware, this is especially true if the application will be performing memory copies involving 2D or 3D objects (whether linear memory or CUDA arrays).” [from the Reference Manual]

This, in fact, solved my problem of the cudaErrorInvalidValue when I tried to cudaMemcpy3D. However, the fact that we use pitched pointer complicates the array access in the kernels. First of all we have to keep track of the data type stored in a pitched pointer (ptr is a void pointer). Furthermore, we have to be careful when dealing with pointer arithmectics because the pitch is, obviously, a byte offset (width of the array).  Here is some sample code that shows how to access pitched pointer data. We assume we want to store float4 data in our 3D array which is of size (width, height, depth). In the kernel we want to access the element (x,y,z).

Host Code:

...
cudaPitchedPtr data;
extent = make_cudaExtent(width* sizeof(float4), height, depth);
cudaMalloc3D(&(d_data), extent);
...

Kernel Code:

__global__ kernel(char* data, size_t pitch, ...) {

...
float4 element = *((float4*) (data + (x*sizeof(float4) + y*pitch + z*pitch*height)));
...

}

The one thing I’d like to point out is the fact that we have to use x*sizeof(float4) for accessing the data. This makes sense because data is a char/byte pointer at the time of the pointer arithmetic. Thus, we have to move the pointer by x*sizeof(float4). This musn’t be done in the y and z direction because the sizeof(float4) is already considered in the pitch (the pitch is the width of the array in byte, probably padded for faster memory access).

The CUDA manual isn’t really precise in telling how to use the pitch with 3D Array (memory obtained by using cudaMallocPitch, cudaMalloc3D, not 3D cudaArrays). So here is a brief summary of how (I think) everything works.

What nVidia means by a pitch is basically the width in byte of an array (2D or 3D). We can specify the width of the array in bytes like this volumeSize.width *sizeof(YOUR_TYPE). With this information we can access any element in a 3D array. We can compute the slicePitch (for a 3D array) which is volumeSize.height*pitch. The slicePitch is the offset in bytes from one slice in a 3D array to the next slice.

Now let’s say we want to access the element (x,y,z) in a 3D array data (which is the base pointer). This is how we can do it:

(myType) (((char*)data)[x*sizeof(myType) + y*pitch + z*slicePitch])

So what we get from the methods cudaMallocPitch, cudaMalloc3D is linear memory which need to be accessed in the way described above.
We need to know how pitches work when we want to copy arrays (when the copy method expects a pitched pointer which we can create with make_cudaPitchedPtr(baseAddress, pitch, width, height)). This is also sufficient for the copy process. Let’s assume we use cudaMemcpy3D. In this case we must specify the extent, in other words the size of our array. All necessary information to execute the copy are now present (it can be ensured that no invalid memory will be accessed) . Each element can be accessed by cudaMemcpy3D as described above. So, this means that we do not need to use the depth (in case of a 3D array) when we make_cudaPitchedPtr.

I had the following problem:

Each CUDA Thread might possible write to any cell in a 3d array in global memory. Thus, some synchronization is required.

The easiest solution is using the atomic operations which are provided by CUDA. Unfortunately, the performance isn’t great. Therefore, I tried improving the performance by implementing a locking mechanism of my own. Simply, using a lock.

The result was that this approach is absolutely useless. I experienced a performance loss of nearly 1000x

I just tried to find a bug in my texturing shader. Unforunatelly, I forgot to set the filtering parameters. As a result, the sampler in the shader always returned 0.

Hopefully I learnt my lesson

REMEMBER: Always set the filtering parameters for textures if you want to do textureing!!!!!