I understand that the TCP itself provides a byte stream connection, but has no idea nor does it care what its payload is ...
Correct.
... and how it is segmented to fit into individual packets that travel on the wire.
Segmenting data into PDUs (TCP segments) and hiding that segmentation from the higher layer are some of TCP's main jobs. TCP derives its maximum segment size (MSS) from the network layer's MTU size.
Observing the process in Wireshark, I can see that the receiver buffers multiple packets that get marked as "TCP segment of a reassembled PDU" and the first incoming entry that follows without said marking is the packet, where all the TCP segments are reassembled into a single TCP data payload.
You are observing fragmentation and reassembly which may seem somewhat similar but are completely independent mechanisms. Make sure you understand each layer's functionality and don't mix up different layers.
In-path fragmentation (IPv4 only) on the network layer is required when an IP packet of some size is to be forwarded over a network that doesn't allow its packet size. The packet is fragmented and the fragments need to be reassembled by the destination node.
Fragmented IP packets are indicated by the presence or absence of the more fragments (MF) flag and its Fragment Offset (FO) value in the IP header.
A common example where TCP segmentation and IP fragmentation can be observed in combination is when the Internet server you've contacted is sending maximum sized segments over its local Ethernet network. Neither your local node nor the other side can see a reduced in-path MTU when both are be using Ethernet with its standard MTU of 1500 bytes. However, that packet size can't be forwarded to you when your WAN link uses PPPoE, eating into the link's MTU. The upstream ISP router needs to fragment oversized IP packets so that they can fit through your WAN link.
Of course that mechanism is inefficient. IPv6 has eliminated in-path fragmentation altogether, requiring each sender to correctly identifying the MTU (via PMTUD) and subsequently the MSS for the destination path.
My first theory was that perhaps when both PSH and ACK flags were set could be used to deduce when the last segment that belongs to the PDU is received,
PSH and ACK are TCP flags entirely unrelated to segmentation or even IP fragmentation.
PSH makes the sending and the receiving stack pass the TCP data on immediately (to the network layer or the application, respectively), without waiting for further data to fill a possibly incomplete (smaller than MSS) segment buffer.
ACK is TCP's acknowledgement mechanism. Unacknowledged data is retransmitted after an appropriate timeout.
My second question is, does the dissector in Wireshark look at the contents of the TCP payload and tries to recognize it in order to determine segments that form a complete PDU?
Wireshark reassembles fragmented IP packets and displays them in whole. That also reassembles a possibly fragmented TCP segment.
Could every transmitted TCP segment that is below the MSS be used as an additional confirmation of the end of a PDU or itself being a non-segmented TCP packet?
No. Each received segment is copied to the receive buffer at the index indicated by its sequence number, regardless of its size, MSS or smaller. Subsequently, a (summarized) ACK is returned to the source.
In latter case, that would mean that the last segment was part of the previous TCP segment.
That doesn't make sense. When segments are ACKed in time, no data is resent.
Perhaps you should take a good look at RFC 9293 before resorting to guesswork.
