Season the chicken on both sides with salt and pepper. In a large cast-iron skillet, heat 3 tablespoons olive oil over medium heat. Add the chicken, skin side down, to the skillet and cook for 15 to 20 minutes. Turn the chicken over and cook for an additional 15 to 20 minutes or until the chicken is cooked through. Remove from the heat and set aside.
Heat the additional 2 tablespoons olive oil in a large pot over medium-low heat. Add the garlic and cook for 5 minutes to soften. Add the tomatoes, kidney beans, and chili powder, and cook, crushing the tomatoes with the back of a wooden spoon, for an additional 5 minutes. Add the water, season with salt and pepper, and bring to a boil. Reduce the heat to low and simmer for 45 minutes. Taste for seasoning and season with additional salt and pepper, if desired. Remove from the heat.
While the soup is simmering, cut the chicken into large pieces.
Divide the soup into individual bowls. Put a piece of chicken in the middle of the soup. Arrange some chili peppers on the soup and sprinkle with the thyme leaves. Serve immediately.