Cut tofu into 1-inch slabs and place on paper towel-lined baking sheet. Cover with another layer of paper towels and place another baking sheet on top. Let sit for 20 minutes. Cut into 1-inch cubes.
Heat oil in a large skillet over medium high heat. Add ginger, shallots, garlic, chile and cilantro stems, and sauté until tender, about 5 minutes. Add mushrooms and sauté until golden brown and tender, about 5 minutes. Season with salt. Stir in curry paste and cook 2 minutes. Pour in coconut milk, scraping up any curry paste with a wooden spoon. Add fish sauce, lime zest and juice. Add tofu cubes and snow peas. Simmer until the sauce thickens slightly and the snow peas are tender, 7 to 10 minutes, stirring frequently. Taste and add more salt and/or fish sauce if needed.
Serve warm with brown rice and a scattering of torn basil and/or cilantro leaves on top.