How organizations guide and facilitate the search for superior options of their members is crucial for value creation. Exploring uncertain options may offer higher returns in the long run, but initially produce outcomes worse than the status quo. This can lead to a premature abandonment of superior options - the well-known “hot stove effect”. Using computer simulations of a learning model, we show that when available options are grouped in categories, and agents generalize more strongly within categories than between categories, the hot stove effect is amplified. We call this phenomenon the ``Hot Kitchen Effect.'' We develop precise predictions about the effect of the number of categories and their structures on the dynamics of quality estimates and test these predictions in experimental studies. Our main finding is that fewer broader categories lead to less exploration of uncertain options and lower cumulative payoff. This suggests that even artificial partitioning of options in categories can mitigate the Hot Stove Effect and improve performance. We discuss the implications of our findings for various streams of research, including research on exploration and organization design as well as on categories.