In order to reset the password of a Workspace you need to reset the password of the user login into the workspace. To do that you have to go to the Directory Service in AWS.

Go to the Directory Service. (Search for Directory Service in AWS console)

Select the directory where the user is, click on Actions > Reset User Password

On the modal that opens up, type the username and the new password. If the username is incorrect you will get a warning that the user cannot be found.