Heat the olive oil in a large cast-iron grill pan over medium-high heat. Add the shishito peppers to the pan and cook for 5 minutes or until they are blistered, flipping them occasionally. Add the Meyer lemon juice, oregano, thyme, and season with salt and pepper. Cook for an additional 1 minute, stirring. Remove from the heat.