Fill a 5-quart pot with water halfway; place on the stove over medium-high heat.
To prepare the dressing: pour the olive oil into a 1-quart mixing bowl. Whisk while slowly pouring in the vinegar to create an emulsion. When all of the vinegar has been whisked into the oil, add the caraway seeds, salt and pepper, and whisk just to combine. Set aside.
When the water comes to a boil, add 2 tablespoons of salt. Add the peeled potatoes to the pot, allow to come back up to a simmer, then turn heat down to low. Let the potatoes simmer for about 20 minutes, or until a paring knife is easily inserted.
Use a slotted spoon to remove the potatoes to a cutting board and allow to cool for 15 minutes. Halve and quarter the potatoes lengthwise, then cut into 1⁄4-inch slices.
Place the warm sliced potatoes in a large serving bowl. Pour the vinaigrette over the potatoes, and toss very gently to combine. Let sit for 10 minutes to allow the potatoes to absorb the dressing.
Add the celery, fennel, carrots, parsley and chives to the bowl, toss gently to combine, and serve.