Since they're close, I could connect them directly?
Yes, you could. However, you need to consider how it makes sense.
If both sides share a common subnet and there are no address collisions then you can just connect both routers to that subnet/segment. Set up a routing protocol between the routers and set static routes to the remote networks (connected to the other router) on each side and you're set.
If both sides are using completely different subnets you could create a new, shared subnet for both routers to connect with. Set up the routes as above.
If both sides use one or more subnets with the same address and you need to keep them separate, there's a problem. The reasonable solution is to jump the hoop and renumber the subnets to remove the duplicity. The ugly and cumbersome approach is to use address translation (NAT).
what's the best way to connect them? Fiber or site-to-site VPN?
VPN connections are encrypted, secure tunnels (mostly) over 'hostile' or untrusted networks like the Internet. If the entire link is under your control there's no need for VPN.
Direct Ethernet connection can go many miles as well (there are extended fiber PHYs for 100 km or even more).
What you should use depends on what you require and whether the already deployed cabling does the job
- Cat-5e copper goes up to gigabit (1G)
- Cat-6A up to 10 gigabit (10G)
- OM2 fiber is good for 500 m with 1G but for only 55 m with 10G
If you deploy new cable you need to consider your requirements for the next (at least) 5 or (better) 10 years. Generally, fiber is more costly and more sensitive but also allows for more bandwidth and reach. Today, you either use OM4, OM5 or go directly to OS2. Whatever kind you deploy, hire a professional to do it.