Capsaicin, and related compounds known as capsaicinoids, give chilli peppers their heat when they are eaten. The capsaicin in chilli peppers excites pain receptors on your tongues, making chilli taste 'hot'.
What makes chillies hot? Capsaicin is the main substance in chilli peppers that provides the spicy heat. It binds to receptors that detect and regulate heat (as well as being involved in the transmission and modulation of pain), hence the burning sensation that it causes in the mouth.