The flow label is a (potentially random) number to help identify flows, ie. series of packets belonging together, not necessarily connection-oriented.
The flow label is placed in the IPv6 header so that networking devices don't have to look into and interpret L4 headers. It is not used for holding a connection together nor can it be used that way.
The IPv6 flow label is specified in RFC 6437:
From the viewpoint of the network layer, a flow is a sequence of
packets sent from a particular source to a particular unicast,
anycast, or multicast destination that a node desires to label as a
flow. From an upper-layer viewpoint, a flow could consist of all
packets in one direction of a specific transport connection or media
stream. However, a flow is not necessarily 1:1 mapped to a transport
connection.
Traditionally, flow classifiers have been based on the 5-tuple of
the source address, destination address, source port, destination
port, and the transport protocol type. However, some of these
fields may be unavailable due to either fragmentation or
encryption, or locating them past a chain of IPv6 extension headers
may be inefficient. Additionally, if classifiers depend only on
IP-layer headers, later introduction of alternative transport-layer
protocols will be easier.
The usage of the 3-tuple of the Flow Label, Source Address, and
Destination Address fields enables efficient IPv6 flow
classification, where only IPv6 main header fields in fixed positions
are used.