coming from a parliamentary democracy (legislatures here technically do have the power to pardon but it's very rarely exercised) I find the practise to be extremely weird.
It essentially inserts an insane amount of arbitrariness and personal favouritism into what is supposed to be an impartial legal process. Given how often it has been abused for political purposes in the US it seems particularly problematic.
The Framers of the Constitution believed that they could thwart the rise of political parties by having a President and legislature who were elected completely separately and would be locked in a power struggle against each other. The pardon power is to prevent legislative overreach -- if the legislature made jaywalking a federal crime with a 10 year jail sentence, the President (or the jury) could simply refuse to allow this law to be enforced.
Despite the high profile political cases, the majority of presidential pardons are given to people who have sat in jail for a long time on crimes that are not at the highest level of turpitude such as murder/rape/kidnapping.
In order to avoid the appearance of impropriety, the Office of the Pardon Attorney was created, along with a process for reviewing and vetting pardon requests.
Historically it has not been quite as arbitrary or controversial as one might think.
It essentially inserts an insane amount of arbitrariness and personal favouritism into what is supposed to be an impartial legal process. Given how often it has been abused for political purposes in the US it seems particularly problematic.