An unmanaged switch doesn't use/care for/understand IP addresses at all.
A managed L2 switch uses IP addresses for management only. Some L2 switches also support limited L3/IP functionality like ACLs. L3 switches use IP addresses for L3 forwarding = routing as well.
When computer A on a local network wants to communicate with computer B for example through the unmanaged switch, does the switch simply just forward the Ethernet frames to computer B?
Yes. A basic switch forwards frames by destination MAC address only. Whether higher layers use IPv4, IPv6 or anything else doesn't concern the switch.
If so, how does it know the destination MAC address initially?
A switch ("multiport MAC bridge") learns locations of MAC nodes from the source addresses they use in the frames they send.
If no frame has been sent from a node, its MAC is initially unknown and the switch needs to mimick a repeater hub by flooding the unknown frame to all its ports but the source port.
Initially when computer A only knows about the computer B's IP address?
For IPv4, broadcast-based ARP is used to resolve an IP address to its corresponding MAC address (for MAC-based networks): A broadcasts an ARP request with B's IP address, B replies with its interface MAC address. IPv6 uses multicast-based NDP.
Does the switch internally keep track of IP addresses matching the MAC addresses?
No. A switch only cares for MAC addresses. Matching IP addresses to those MAC addresses is up to the hosts (or more generally up to the L3 nodes, including routers).