Then I found that my Android phone connects to the new one automatically without asking me password.
Is this some kind of Wi-Fi specification?
First, a PSK is not a password. This is a common misconception.
A PSK does not provide any sort of authentication. Rather, it is a known value by the station and AP to generate and securely (at least from outside sources without the known value) exchange keying material for use in encrypting further traffic.
To be clear, WPA/WPA2-Personal does not authenticate, it only encrypts traffic.
As such, the behavior you describe is normal 802.11 operation. Your Android device was able to connect to the SSID it was configured for and then with the same PSK, it was able to successfully negotiate encryption with your new access point.
When a device connect to a wifi router, what else does it look beside SSID? for example MAC address? so even if a new device with same SSID comes up, it still makes the new device a different one?
Most client devices only look at the SSID. Some can be configured to look instead (or additionally) to the BSSID, which is a 48-bit MAC address.
Limiting the scope of a client's ability to connect to a specific BSSID will either prevent roaming (between APs in a multi-AP deployment or between radios in a dual band deployment with shared SSIDs) or require multiple configuration settings to add each BSSID in the environment to the client device.
does it a security issue?
No, this is normal operation. You just need to understand the limitations of the technology in use. WPA/WPA2-Personal provides encryption to your wireless communications. That is it.
It is not meant to provide any sort of benefit that authentication normally provides nor to validate the network to which you are connecting. Additionally, any device with the PSK that also captures the WPA/WPA2 encryption handshake (i.e. when the client device connects to the wireless network) will also be able to decrypt all the data.
Does this make it insecure. No, rather it is just providing the level of security for which it is designed.
If you want a more secure option, then WPA/WPA2-Enterprise is the way to go. Depending on the deployment choices, it can provide unique encryption material for each connection, validate the user and/or device connecting to the network and validate that the network to which you are connecting. This does come as a trade off as it is more complex on both sides of the connection and can require user education to maintain security. This is a normal security vs. ease of use trade off.
Generally speaking, WPA/WPA2-Personal is sufficient for most small deployments.
Note: some portions of this answer have been copied from my own answers to similar questions elsewhere on the SE network. As they are my own answers, I am not citing them.