How to Upgrade Underscore JavaScript library without causing malfunctioning of the CMS

You will need to have full source code license for this and then also do a full round testing of the CMS (which usually takes more than 200 man-hours)

@Juraj Ondrus thanks for your response; so does that mean that the Kentico Support we pay for yearly doesn't cover this?If it does, how do I get such support from Kentico?

Support maintenance does not cover customizations nor this kind of updates - as mentioned, this is really a pretty big thing and this needs to be done also in the source code by the development team and then everything needs to be tested. We do not offer such services.
Moreover, our security team is watching these and so far we are not using the vulnerable parts of these libraries in our implementation.