Basically, my question is that for these official ports, can I still let my own application use these ports for other data exchange purposes? What could be the downsides if I choose to do so?
Yes. You can use any port for anything (pretty much with a few exceptions) provided you can configure the applications port. Of you can forward it in your router, you can forward any external port to any internal port usually (if you needed a workaround.)
In fact, FTP ports are commonly used for exploits as many firewalls will have those ports open and since FTP servers send and receive files, and often have weak authentication mechanisms, they can be used to upload malicious code or proxy a remote attack.
But the answer is yes. Yes you can.
Quick Edit: Although sometimes software will prevent you from using certain ports, if you're writing your own application, that would not be an issue (plus you can always forward the port as I mentioned above, of necessary.)
There really aren't such things as "reserved ports" like there are with IP addresses, just commonly used ports (such as port 80 for http, 443 for https, 8080 for http proxies, etc.- but I can put an http web server on port 59623 if I want- although some browsers may call it a security risk and not load it). Some very low numbered ports are used for low level ethernet protocols, such as ICMP, Echo, Discard, etc. These ports will likely already be in use and therefore difficult to assign (but not impossible.)
If a port is already assigned, you will need to first kill the program that is using it before you can use it for your own program.
Also, on many OSs, ports below 1000 (or something in that area, certain distributions it's below 10000) are blocked from assignment by non-root users or unpriviledged apps. This is the case in most linux distros (you can assign a low number port, but only with SU priviledge.) This is also true of Android OS (which is just a heavily modified linux kernel.)
The principal downside of using, say, port 20 for web, is confusion. If you're in charge of a computer, you can run whatever you like on whatever port you like. (So long as your software is configurable and within the rules that running processes will stop new ones from using the same port.)
The administrative "reservation" of port numbers to given services is really extremely informal, and you have to remember how fluid everything is.
The early major services got their "well known port numbers" by Cerf and Postel surveying what everybody was using in 1972: "We would like to catalog other sockets which are supposed to be well-known, so we would appreciate having a note or phone call ..." (RFC 322). Later this became the Assigned Numbers series of RFCs.
It's worth noting that in the Assigned Numbers RFC (last was 1700 before it went database) there are hundreds of so-called "well-known" ports most networking people have never used or even heard of, either they are obsolete, obscure, overrun by more-sensible methods or indeed, they just reserved a number but they were never really implemented outside of one or two sites. (The same is true for protocols; personally I think I've only used about half a dozen of the protocols, and recognise only one or two more.)
As an aside, from a security point of view, this is why firewalls have to inspect the content of traffic to make an algorithmic guess about the traffic. If your policies decide that HTTP to particular sites is prohibited, how do you actually enforce it? You can't simply look at port numbers. (You can't just look at IP addresses either, because of proxies. And you might not be able to look at the GETs because of HTTPS. In short it might be very difficult.)
It easy to criticise this as chaos. But the ease and openness of implementing the internet protocols, and the ease of experimenting with them and devising new things is precisely what made the internet take off.
Email and News was considered fabulous until those physicists started messing about with document transportation, eventually producing what became the all-conquering web, which did not get a reserved port number until 1992 (RFC 1340, absent in RFC 1060 of 1990).