I think we can produce a significant speedup in our execution of LCA. Since most of the neurons in any LCA computation are below threshold for most of the display period, perhaps we could just ignore them (with occasional reality checks) and work entirely within the much smaller subset of neurons that are either active or only slightly below threshold. I believe it should be possible to exploit this idea to obtain a big win with relatively minor changes to the code.
It seems that the basic strategy would thus be:
- Make a list of "eligible" neurons whose membrane potentials are positive (V >= 0). One could also pick a different threshold value for eligibility but 0 seems like a natural choice.
- For each time step within a given display period 'i' evaluate whether i < T1 || mod(i,T2) == 0
if the above condition is false, only receive input to and update the neurons in the eligible list. Since the receive step is typically performed on the GPU from the post perspective, this should be a relatively simple change. the neuron index would be drawn from the eligible list rather than from the list of all neurons. note that updates are typically performed on the GPU as well and would also benefit from a much smaller list of targets to update.
if the above condition is true, then receive input to and update ALL neurons. the idea is that once the network has settled into a good enough approximation to the optimal sparse code, we only have to occasionally check whether a neuron has been promoted to the eligible category or alternatively demoted out of the eligible population.
in the above, T1 would be set to the smallest value at which eligible neurons could be identified with reasonable accuracy and T2 would be set to the largest value we can get away such that the final answer was independent of T2 (to within some tolerance).
Since we are somewhat CPU bound right now, the above scheme would probably have the biggest impact on Trinity. However, by taking advantage of sparsity even when receiving from the post perspective, we might also be able to obtain competitive performance on CPU only machines, which are much cheaper in the AWS cloud right now compared to GPU machines.
The above scheme should also speed up our computations on the GPU considerably, which can only help our overall performance.
It's also possible that our timing information is too crude to resolve what's actually happening during a display period. It might well be that we're only CPU bound during the initial portion of the display period when the activity is not very sparse due to the recent image flip. Later in the display period, once the activity is more sparse, we might might be more GPU bound, at least relative to the initial portion of the display period. Thus, we might get a bigger win even on a GPU machine than our current timing results indicate. Right now, we're about equally divided between GPU and CPU based convolutions, so speeding up the GPU convolutions should still give us a decent speedup.
Of course, the above behavior would have to be under the control of a "useEligibilityListFlag" since we're not always doing LCA.
I think we can produce a significant speedup in our execution of LCA. Since most of the neurons in any LCA computation are below threshold for most of the display period, perhaps we could just ignore them (with occasional reality checks) and work entirely within the much smaller subset of neurons that are either active or only slightly below threshold. I believe it should be possible to exploit this idea to obtain a big win with relatively minor changes to the code.
It seems that the basic strategy would thus be:
if the above condition is false, only receive input to and update the neurons in the eligible list. Since the receive step is typically performed on the GPU from the post perspective, this should be a relatively simple change. the neuron index would be drawn from the eligible list rather than from the list of all neurons. note that updates are typically performed on the GPU as well and would also benefit from a much smaller list of targets to update.
if the above condition is true, then receive input to and update ALL neurons. the idea is that once the network has settled into a good enough approximation to the optimal sparse code, we only have to occasionally check whether a neuron has been promoted to the eligible category or alternatively demoted out of the eligible population.
in the above, T1 would be set to the smallest value at which eligible neurons could be identified with reasonable accuracy and T2 would be set to the largest value we can get away such that the final answer was independent of T2 (to within some tolerance).
Since we are somewhat CPU bound right now, the above scheme would probably have the biggest impact on Trinity. However, by taking advantage of sparsity even when receiving from the post perspective, we might also be able to obtain competitive performance on CPU only machines, which are much cheaper in the AWS cloud right now compared to GPU machines.
The above scheme should also speed up our computations on the GPU considerably, which can only help our overall performance.
It's also possible that our timing information is too crude to resolve what's actually happening during a display period. It might well be that we're only CPU bound during the initial portion of the display period when the activity is not very sparse due to the recent image flip. Later in the display period, once the activity is more sparse, we might might be more GPU bound, at least relative to the initial portion of the display period. Thus, we might get a bigger win even on a GPU machine than our current timing results indicate. Right now, we're about equally divided between GPU and CPU based convolutions, so speeding up the GPU convolutions should still give us a decent speedup.
Of course, the above behavior would have to be under the control of a "useEligibilityListFlag" since we're not always doing LCA.