I'd like to add to dmckee's answer: he is absolutely right: people do use very sloppy language and in problems like this such language can do your head in. His pithy summary of why energy conservation is not violated can't be bettered. What people are of course talking about when they say there's an EMF of $V$ volts around a loop $\Gamma$ is that $\oint_\Gamma \mathbf{E}\cdot\mathrm{d}\mathbf{r} = V$ and, since the potential difference $\Delta V$ between two point $A$ and $B$ in a conservative field is independent of path and given by $\Delta V = \int_A^B \mathbf{E}\cdot\mathrm{d}\mathbf{r}$, people make a rough (and in general wrong, in the way dmckee shows) analogy between these two similar-looking integrals. But there are two particular reasons why people do this.
1. Approximate Potential Functions Outside Compactly Supported Magnetic Fields
Sometimes we can indeed define a potential with branching behavior and the points of "low" and "high" potential can be many different pairs: they are defined by a "convention" which amounts to the siting of a "branch cut" and we have considerable freedom in siting this cut as I am about to explain. Suppose we have a magnetic field $\mathbf{H}(\mathbf{r})$ pointing into the page and well confined to some compact region, say the darkly shaded disk $\mathcal{M}$ in my drawing below. For example, this could be the cross section of a high permeability magnetic circuit in a motor or transformer. Suppose further that frequency is low in the sense that the wavelength is far greater than dimensions of the system under consideration, then the displacement current term in Ampère's law can be neglected, so the total magnetic flux through the loop is negligibly changed by the time varying electric field that the time varying flux produces. Then, outside the shaded region we have $\nabla \wedge \mathbf{E} = \mathbf{0}$ so that electric field can indeed be representable by a potential $\phi_E$: by Stokes's theorem, $\oint_\Gamma \mathbf{E}\cdot\mathrm{d}\mathbf{r} = 0$ so that $\Delta V(A,B) = \int_A^B \mathbf{E}\cdot\mathrm{d}\mathbf{r}$, where $\Gamma = \partial \Sigma$ is a piecewise smooth boundary of any simply connected region $\Sigma$ outside $\mathcal{M}$, is essentially independent of the path between $A$ and $B$: two paths linking $A$ and $B$ yield the same value for $\Delta V(A,B)$ as long as one can be continuously deformed into the other without the passing through the magnetic field; more precisely put: as long as the two paths belong to the same homotopy class with respect to the multiply connected region $\mathbb{R}^2 \sim \mathcal{M}$ (i.e. the plane with a hole in it), or more intuitively put: as long as the loop formed by the two paths does not wind around $\mathcal{M}$. So we can define a potential function by choosing any point fixed point $A$ outside $\mathcal{M}$ and defining $V(B) = \int_A^{\mathbf{r}_0} \mathbf{E}\cdot\mathrm{d}\mathbf{r}$ with the further proviso that the path linking $A$ and $B$ must "stay on the same side of $\mathcal{M}$ as $A$: the way we make this proviso precise is just as we do with complex functions that are holomorphic everywhere aside from at some branch point: we define a "Branch Cut" linking the origin to infinity with the behest that our path defining $V(B)$ must not go through the branch cut. I have shown one possible branch cut as the dashed line (the positive $x$-axis). It matters neither where $A$ is nor the exact branch cut, as long as the two stay the same in any discussion. Once we do this, $V(\mathcal{r})$ is uniquely defined and indeed $\mathbf{E}(\mathcal{r}) = \nabla V(\mathcal{r})$. This potential is like any other normal electrostatic potential aside from its branching behavior: by thinking about the big path $\Theta$ it is easy to see that the potential difference between any pair of points outside $\mathcal{M}$ immediately either side of the branch cut (e.g. $P_1$ and $P_5$ or $P_2$ and $P_4$) does not depend on the choice of points is $\mathcal{E} = \mathrm{d}_t \,\left(\int\int_\mathcal{M} \mathbf{B} \cdot \hat{\mathbf{z}} \mathrm{d}x\,\mathrm{d}y\right)$, or what people call the "electromotive force" or "voltage" around the loop. It doesn't matter where the branch cut is, so the points of "high" and "low" potential in the wording of your question can be any pair like $P_1$ and $P_5$ on either side of any consistently defined branch cut. The analogy with holomorphic functions of a complex variable can be made even more complete: suppose in this case that $\mathcal{M}$ is a circular disk centred on the origin, then the complex potential is $\Psi(\zeta) = - i \frac{\mathcal{E}}{2 \pi} \log \zeta$ where $\zeta = x + i y$, the real everyday potential is $V(\zeta) = \operatorname{Re}(\Psi(\zeta))$, the electric field at $\zeta$, both magnitude and direction, is represented by the complex number $(\mathrm{d}_\zeta \Psi)^*$ and the electromotive force around the loop $\mathcal{E}$ is $\mathcal{E}=\oint_C \Psi(\zeta) \,\mathrm{d}\zeta$ for any loop containing $\mathcal{M}$. For any arbitrarily shapen region $\mathcal{M}$ where the magnetic field pierces the page and is orthogonal to the page, one can define a corresponding complex potential (by the Riemann mapping theorem).

2. The Analysis of Electrical Machines, particularly Generators
Now we imagine our loop with EMF $\mathcal{E}=\mathrm{d}_t \,\left(\int\int_\mathcal{M} \mathbf{B} \cdot \hat{\mathbf{z}} \mathrm{d}x\,\mathrm{d}y\right)$ around it. We might then imagine a wire loop $\Gamma_2$ wound around the magnetic flux as in the drawing below and brought to a close pair $P$ so that the pair can leave the region steeped in magnetic fields and be connected to a remote load: at the top of the drawing I have imagined it to be a capacitor. Moreover, the voltage across the capacitor's plates is $\mathcal{E}$, the electrostatic field within the capacitor is a genuinely electrostatic field with a well defined electric potential (now without branch cuts and the capacitor's plates are equipotential surfaces. Notice how for this interpretation to work, the magnetic field doesn't have to be confined tightly within it as it was for the examples above: there can be magnetic field all around it. The only requirement is for the pair to run off to an exterior circuit where the magnetic field has no influence. The load, of course, does not have to be a capacitor: it could be any electrical circuit making up a load of general impedance $Z$. So now the EMF is the potential difference between the poles of the ideal voltage source that we use to represent the generator, and the loop's EMF has thus taken on the electrical engineer's meaning of source voltage in a general circuit. Which brings me to one last interpretation of the EMF which might help you to visualize things. Imagine that the capacitor circuit is shrunken so that the capacitor is inside the region influenced by the magnetic field and so we have a loop looping directly from the capacitor's terminals. Equivalently, we could imagine the loop to be a split ring where the two ends of the split ring are brought very near together. This is what we would get if the capacitor's capacitance were very small, so that even tiny charges on its plates generate huge voltages. What would happen in such a case? Well, of course a current could not flow, because the ring is split. Why? The EMF causes charges to amass on either sides of the split until the EMF is opposed by the electrostatic field arising from the charges, and current does not flow (or at least very little does, there is a tiny trickle over each AC cycle to re-arrange the charges to oppose the EMF). With a little thought, you can understand that the potential difference arising from the charges gathering at the split is precisely the EMF $\mathcal{E} = \mathrm{d}_t \,\left(\int\int_\mathcal{M} \mathbf{B} \cdot \hat{\mathbf{z}} \mathrm{d}x\,\mathrm{d}y\right)$. The points of low and high potential are then the plate potentials of the capacitor, which plays the role of a physical branch cut.
