This was in some custom code I was writing, but I'll illustrate it with the Upload users function which displays exactly the same issue...
1. Go to Users > Accounts > Upload Users
2. Click Choose a file, then (in the picker) Upload and pick any file and click "Upload this file"
3. Name of uploaded file is correctly shown under Choose a file button
4. Change your mind (wrong file), repeat the above process with a different file
5. New/different file is now shown under the Choose a file button
6. Ooops! You were correct the first time, repeat again and go back to the original file
7. Try to repeat the process with the original file, you can't "File name already being used, please use another name"
It's not end-of-the-world and the workaround is obvious, but it's a bit shoddy and will confuse users who find the filepicker confusing enough already.