To utilize all textual, audio, and visual information to predict the severity of depression., we proposed a hierarchical recurrent model. Our method contains two hierarchies of bidirectional long short term memories (LSTM) to fuse multi-modal features and predict the PHQ-8 score. We are the runner-up in the AVEC2019 challenge of depression detection.