Once you have the file, you cannot just "open" it. You must use a sideloading tool to sign and install it.
One way to download an older version of YouTube IPA is to use a third-party app store, such as AppPure or AppCake. These app stores offer a wide range of iOS apps, including older versions of YouTube. youtube ipa download old version