(By the way, I have noticed that there are no votes to reopen the question from the community. Asking a question on a community that does not actually want the question means that the question will languish forever, with no answer. That is not what you want, and we are trying to get you to the right place, with the right question, to get an answer to your question.)
Teun has given you reasons why your question was closed as off-topic for Network Engineering, but let's look at it in detail:
I have two houses with internet connections
Do you have business Internet service rather than residential internet service? This matters a lot more than you may think. Residential service could be using CGN, which then makes it impossible to do what you want the way you want. Residential service has a clause in the contract prohibiting running services from the residential network. Also, residential service even without CGN is usually subject to having the IP address change, breaking any connections. Those are some reasons that residential/home networks are off-topic here.
I would like to be able to transparently access the file server from
computers at my other house (House B), as if the file server were
connected directly to both house's local area networks.
That is simply off-topic here as host/server questions and questions about protocols above OSI layer-4. You can ask about hosts/servers and protocols above OSI layer-4 on other SE sites, as has been recommended to you.
I am trying to get an opinion on the cheapest reliable way to achieve.
Primarily opinion-based answers are explicitly off-topic on almost every SE site. Also, questions with an "-est" adjective (cheapest, fastest, best, etc.) are almost never suitable for SE sites, as they lead to opinions and arguments, and the "-est" part is often short-lived (the fastest processor usually holds that title for only a few months, but SE sites are archives that are supposed to last many years).
The router/modem at House A supports OpenVPN natively, however I would
need to replace the router/modem at House B in order to get a model
that supports OpenVPN - replacing this modem is something I am
avoiding doing because this router also runs a VoIP phone and has been
A "router/modem" is a consumer-grade device that is explicitly off-topic here. Those are used for residential (off-topic) Internet service, often supplied by the ISP. Also, OpenVPN is off-topic because the manufacturer does not offer optional paid support.
Is this a feasible thing to achieve?
That is a question looking for a simple yes/no answer, and not particularly useful for SE sites. The answer is, "Yes, that is fairly easy and done every day," but that is from the perspective of a "professionally managed network in a business environment," which is what is on-topic for Network Engineering. For a residential network, such as you have described, it can be much more difficult to impossible, but we simply do not have enough information to answer the question.
Your comments to Teun's answer:
1) nowhere does it say being a doctor's surgery makes it on-topic for
a network engineering site
A doctor's office could be on-topic for Network Engineering, as long as the question meets the on-topic criteria, but none of the off-topic criteria, list on the What topics can I ask about here? page. The first sentence reads:
Network Engineering Stack Exchange is for asking questions about
professionally managed networks in a business environment.
Unfortunately, the networks you have described are far from professionally managed.
There is also a note in the introduction on that page:
(Note: All questions about RESIDENTIAL/HOME networking and
CONSUMER-grade equipment, are explicitly OFF-topic.)
Your original question does nothing to dispel the notion that your networks are residential networks using consumer-grade devices. On the contrary, it comes right out and says that is what the networks and network devices are.
2) do the models of peripheral devices affect network configuration of
a separate network device? How would you possibly conclude that
consumer-grade equipment is used...
Yes. As I explained above, your "router/modems" are consumer-grade devices, often provided by the residential ISP. A proper business-grade router for which the manufacturer offers optional-paid support (a requirement if you read the link about what is on-topic for Network Engineering) usually has the ability to create a tunnel between two sites (it may require a license upgrade to encrypt the tunnel).
3) the question is asking how to expose a network interface in a
particular way, and the context of it being a file server merely sets
the scene - I would expect an answer to apply to many scenarios
The answer I gave you (yes, for a professionally managed network, and maybe, for a home network) does apply to many scenarios, but it is simply too broad to be useful for this type of site. Remember that SE sites are not help forums, but Q&A sites that form an archive of questions and answers. Such questions that have such broad or general answers, or answers that are primarily opinion-based are not suitable for SE sites.
4) the word 'device' is not a product, again this is probably a
There are network devices (routers, switches, etc.) that are on-topic for Network Engineering, as long as they meet the caveat from the above link:
hardware that has a paid support option from the manufacturer
(enterprise/provider class products, some small business class
There are also many devices that are not on-topic for Network Engineering (consumer-grade network devices, and end-devices such as PCs, printers, servers, mobile phones, etc.).
I also would have closed your question as off-topic as it was written. The question, as it is written, simply does not meet the requirements for Network Engineering, nor many other SE sites. For example, it could be on-topic for Server Fault, but it would be closed as too broad and soliciting primarily opinion-based answers, even if it did not mention the home networking aspect. It may be on-topic for Super User, but again, it is simply too broad and soliciting opinions. With some work, you could rehabilitate the question to meet the criteria for one of those sites, but it is unlikely that you could modify it to be on-topic for Network Engineering to get an answer that would help you, and that means it would simply sit unanswered, so we close it as off-topic and try to guide you to a better question in a better place to ask it.