LESSWRONG
LW

All of joba2ca's Comments + Replies

How to measure FLOP/s for Neural Networks empirically?
joba2caΩ020

Thanks for the detailed analysis, especially regarding the weird behavior of nvtx. I found the comparison of profiling approaches quite helpful.

Are you aware of any updates to your the profiling methods regarding their precision since the time of your analyses?

Reply
2Marius Hobbhahn
I have not tested it since then. I think there were multiple projects that tried to improve profilers for PyTorch. I don't know how they went.