port speedups identified in test code to PetaVision

[garkenyon]

the following is cut and pasted with minor edits from a report by Boram Yoon and Scot Halverson based on their study of a ~100 line test code intended to model the convolution patterns in PetaVision on a representative network.

numbers appearing as **** have been omitted to comply with possible NDA restrictions:

---

the speedups identified below refer to the
~100 lines mini-app.

The updates are pretty simple but cannot be copy-and-pasted to the original PetaVision
code.

Identified speedups are summarized below

1. Aligned memory allocation

Intel compilers have a memory allocation function that allows aligned
memory allocation: _mm_malloc()
In the mini-app, we have replaced malloc(SIZE) to _mm_malloc(SIZE, 64) for
a 64-byte aligned memory block:

// float _data1 = (float *)malloc((size_t)(numNeurons/shift_period_shift

• sy_yPatchSize)_sizeof(float));
  float _data1 = (float
  *)_mm_malloc((size_t)(numNeurons/shift_period_shift +
  sy_yPatchSize)_sizeof(float), 64);

In real code, the calloc() should be replaced by the _mm_malloc, only for
INTEL compilers, and initialized by 0.

In addition to the memory allocation, it needs a #pragma tells the vectors
in a for-loop are aligned, as follows:

pragma vector aligned

for (int k = 0; k < numPerStride; k++)
{
dv += a[k] * w[k];
}

The pragma should be placed in the most-inner heavy loops.

In the mini-app, the aligned memory improved by **% (30%) on KNL (Haswell)
for deliverPost..., but did not improved for deliverPre... routine.

1. Tiling size

In deliverPost, Jeff has reordered and tiled the for loop by

for (int feature = 0; feature < nfp; feature++) {

for (int idx = feature; idx < numNeurons; idx += nfp) { ... }

}

However, we found that different tile size is faster for the input params
file Jeff has made for the Hackathon:

int tile = nfp*4;
for (int feature = 0; feature < nfp; feature++) {
for (int idx = feature; idx < numNeurons; idx += tile) { ... }
}

The new tile size improved the performance by a factor of *** on KNL, and
10% on haswell.
I am not sure if 4*nfp would be the most efficient tile size for different
input params files.

1. Loop reordering for deliverPre

Jeff has reordered the loop structure for deliverPost. We has made similar
reordering for deliverPre case

     // Weight
     PVPatch * weights0 = getWeights(0, arbor);
     for (int y = 0; y < weights0->ny; y++) {

pragma omp parallel for schedule(static)

endif

  for (int idx = 0; idx < numNeurons; idx++) {
     int kPreExt = activity->isSparse ? activeIndicesBatch[idx] : idx;

     // Activity
     float a = activityBatch[kPreExt] * dt_factor;
     if (a == 0.0f) continue;

     // gSyn
     pvdata_t * gSynPatchHead = gSynPatchHeadBatch;

ifdef PV_USE_OPENMP_THREADS

     if(thread_gSyn) {
        gSynPatchHead = thread_gSyn[omp_get_thread_num()];
     }

endif // PV_USE_OPENMP_THREADS

     pvgsyndata_t * postPatchStart = gSynPatchHead +

getGSynPatchStart(kPreExt, arbor);

     // Weight
     PVPatch * weights = getWeights(kPreExt, arbor);
     const int nk = weights->nx * fPatchSize();
     const int ny = weights->ny;
     pvwdata_t * weightDataStart = get_wData(arbor,kPreExt); // make

this a pvwdata_t const *?

        float * v = postPatchStart + y * sy;
        pvwdata_t * w = weightDataStart + y * syw;
        for (int k = 0; k < nk; k++) {
           v[k] += a * w[k];
        }
     }
  }

Here the only difference is that we moved y-loop to outside of the
threading region

for (int y = 0; y < weights0->ny; y++) {...}

Assuming that ny is the same for the neurons:

PVPatch * weights0 = getWeights(0, arbor);
for up to weights0->ny

This loop re-ordering made factor of *** (factor of 3) performance
improvement for ny=10 case on KNL (Haswell). In the real calculation with
the random-data params file Jeff has made for the Hackathon, ny is varying
from 1 to 10, and the overall improvement was factor of 2 on Haswell; we
expect about *** on KNL.

1. OpenMP scheduling

After applying all improvements, we found that static scheduling is better
for deliverPost. In case of deliverPre, however, dynamic scheduling with
chunk size of "nk" was faster on Haswell:

#pragma omp parallel for schedule(dynamic,nk)

Performance improvement from the dynamic scheduling was about 20% on
Haswell for the mini-app.

Those are the four updates that need to be implemented on the petavision
code. The optimization is based on the trinity branch.

[peteschultz]

Regarding memory allocation alignment, there are pvMalloc and pvAlloc macros defined in utils/PVLog.hpp. We should convert the malloc and alloc calls in the src hierarchy to pvMalloc and pvAlloc, and then put the swap of _mm_malloc() for malloc() in PVLog.hpp