I believe this is well-known to those working in regular/inverse semigroups. So well-known in fact that one can even find proofs on the web, e.g. see this proof on PlanetMath. Alas, I cannot recall the history of this result, though I vaguely recall reading something about such decades ago.
For completeness, I reproduce below the proof presented on PlanetMath (slightly edited).
Theorem $\ $ A non-empty semigroup $S$ is a group if and only if for every $x\in S$ there is a unique $y\in S$ such that $xyx=x$.
Proof $\ $ Suppose that $S$ is a non-empty semigroup, and for every $x\in S$ there is a unique $y\in S$ such that $xyx=x.$ For each $x\in S,$ let $x'$ denote the unique element of $S$ such that $\,xx'x=x.\ $ Note $\,x(\color{blue}{x'xx'})x=(xx'x)x'x=x\color{#C00}{x'}x=x,\,$ so, by uniqueness, $\color{blue}{x'xx'}=\color{#C00}{x}',$ and therefore $\color{blue}x = \color{#C00}{x}''.$
For any $x\in S,$ the element $xx'$ is idempotent, by $(xx')^2=(xx'x)x'=xx'.$ As $S$ is nonempty, we infer that $S$ has at least one idempotent. If $i\in S$ is idempotent, then $ix =ix\color{#0A0}{(ix)'}ix=ix\color{#C00}{(ix)'i}ix,$ so, by uniqueness, $\color{#C00}{(ix)'i}=\color{#0A0}{(ix)'},$ hence $(ix)'=(ix)'(ix)''(ix)'=\color{#C00}{(ix)'i}x(ix)'=\color{#0A0}{(ix)'}x(ix)',$ so, by uniqueness, $x = (ix)''=ix.$ So every idempotent $i$ is a left identity, and, by a symmetry, a right identity. Therefore, $S$ has at most one idempotent element. Combined with the previous result, this means that $S$ has exactly one idempotent element, denoted $e$ . We have shown that $e$ is an identity, and that $xx'=e$ for each $x\in S,$ hence $S$ is a group.
Conversely, if $S$ is a group then $xyx=x$ clearly has a unique solution, namely $y=x^{-1} . $
$\ $ QED