ReLU가 합성된 행렬 곱 연산의 부분 생략을 통한 딥 러닝 모델 추론 시간 개선

초록

Deep learning has expanded its utilization, and large-scale deep learning models containing many parameters tend to perform well. As large-scale models inevitably require many resources and long inference time, reducing the inference time is essential for efficient utilization of deep learning models. We fuse the activation function Rectified Linear Unit and matrix multiplication in the inference process, and reduce the amount of computation by predicting the sign of the output values to be computed in the computational processes. We propose four methods for reducing the computation and derive an optimal method that saves inference time with low accuracy loss by reducing the amount of computation by comparing these four methods.

키워드