Understanding why an architecture crashes or does not perform as expected may be difficult sometimes. Here are some common ways around this:

Use the “_debug” version of the corresponding tool. For example, detect_debug rather than detect will natively print a lot of information about what is happening, including what operations are performed with which input sizes and data ranges. This allows to trace the progression of the input through the network, giving you an idea if something is terribly wrong or not. For example, a min/max range of [0, 0] means that information is not going through correctly starting at the layer corresponding to the first occurrence of this range.

Some tools provide visual feedback of internal processing, given that the display variables are activated. Refer to tools for display variables.

The train tool displays correct and incorrect answers, ordered by lowest and highest energies. This can help determine what the system finds easy and hard to classify and identify problems in the data, for example outliers or groundtruth errors.

Tools based on detection_thread such as detect provide some visual insight about the multi-scale inputs and outputs. Empty inputs means there is a problem at the preprocessing level. If inputs are correct but outputs are empty, there is a problem at the network level. Then you can use the debug outputs to figure out at which layer the information is blocked. To enable all visual feedback for detection tools, use these variables:

display         = 1 # enable/disable display altogether
display_threads = 1 # each thread displays on its own 
display_states  = 1 # display internal states of 1 resolution 
show_extracted  = 1 # show positive input patches
minimal_display = 0 # only show classified input 

One sometimes encounters differences between 2 systems which origins are hard to identify. One way to narrow down these differences is to use the dump_fprop() feature available that is equivalent to a regular fprop() of a module but in addition dumps inputs, outputs and internals in matrix files on the disk.

To dump all values produced by the first sample of a dataset, use the dump option of dsfprop as follow:

dsfprop my.conf -dump