Virtual private networks have emerged from obscurity to become the frequently favored method of connecting private networks. While VPNs gained popularity for enabling the use of the Internet to secure network connections, thereby removing the need for costly dedicated circuits, VPN adoption soared because the technology proved to be relatively straightforward, dependable, and secure.
Believing VPNs are foolproof, however, can create a false sense of security. Following state-sponsored attacks that exploited compromised VPNs to facilitate harmful attacks, organizations were alerted that VPN accounts also require rigorous monitoring and protection.
By implementing proper security measures, VPNs continue to effectively meet a crucial need to securely and reliably link remote employees, branch offices, authorized partners, and other systems. Nonetheless, VPN connection problems continue to arise inevitably.
Many of the VPN connection errors encountered, particularly those powered by Windows server, typically fall into one of four categories:
- The VPN connection gets denied.
- An unauthorized connection is allowed.
Here is how to address these common VPN connection errors powered by Windows Server.
Dealing with the Windows Server Routing and Remote Access console
Occasionally, when a VPN is set up using a Windows Server, connection issues may arise, even if the connection previously functioned correctly. Troubleshooting often involves interacting with the Windows server’s Routing and Remote Access console snap-in tool, where Microsoft consolidates many VPN configuration settings.
The Routing and Remote Access snap-in is located within the Microsoft Management Console, also known as the MMC. There are various methods to access the MMC. You can choose the console from the Start menu’s Programs options, within the Administrative Tools folder in the Windows server’s Control Panel, or by typing ‘mmc’ at a command prompt. Alternatively, you can access the MMC by pressing the Windows key and the letter R simultaneously, entering ‘mmc’, and pressing Enter.
While the actual user interface and menu options may slightly vary between specific server versions, administrators should be able to navigate the different consoles — whether dealing with an older version or the latest Windows Server 2022 iteration — using the same method.
How to rectify the four main issues with unsuccessful VPN connections
1: The VPN connection gets denied.
The most common VPN issue is having a VPN client’s connection rejected. The reason this problem is prevalent is that numerous issues can lead to a connection rejection.
If client connections are rejected by the Windows server-powered VPN, the first step is to verify that the Routing and Remote Access Service is running on the Windows server. You can verify this by opening the Windows server’s Services console, which can be accessed by clicking Start | Control Panel | Administrative Tools | Services. With the Services console open, navigate through the list of services
Visit the Routing and Remote Access entry to guarantee its operation.
The Services console reveals the status of the Routing and Remote Access entry. By highlighting the Routing and Remote Access entry in the Services console, you have the option to initiate the Service or right-select the entry and choose to Restart. In case the RRAS service was configured as Manual or Disabled, you can access the entry, modify the Startup Type to Automatic, and proceed to Start and confirm by clicking OK.
Upon verifying the RRAS service is functioning, it’s advisable to run a connectivity test by pinging the VPN server initially by IP address and then by its fully qualified domain name. If errors are encountered, it is likely a DNS issue, which requires attention to resolve.
In scenarios where the VPN server pings are successful, and connection difficulties persist, investigate a potential authentication mismatch. At times, the VPN client and VPN server are configured to utilize varying authentication methods.
Validate if an authentication error is the concern by launching the server console. Another pathway to access the MMC is by entering Control+R to open a command prompt, then typing mmc and pressing Enter or clicking OK.
With the console opened, navigate to the Routing and Remote Access entry. If the entry is not visible, tap on File, opt for Add/Remove Snap-in, choose the Routing and Remote Access alternative from the list, click Add, then OK.
Upon adding the Routing and Remote Access snap-in, right-click on the VPN server and navigate to Properties. Subsequently, review the Security tab to ensure the authentication method. Commonly, Windows Authentication is used, although an alternative option such as RADIUS may be in place. Validate that the VPN client aligns with the specified authentication method within the Security tab.
Additional items for examination
Primarily, the reviewed elements are typically attributed to most VPN connection refusal occurrences. Nevertheless, other rudimentary aspects should also be correctly configured.
For instance, if the Windows Server hosting the VPN has not joined the Windows domain, authenticating logins will not be possible. Initially, connect the server to the domain.
IP addresses are another elementary factor necessitating apt administration. Typically, each Web-based VPN connection employs two distinct IP addresses for the VPN client computer. The first IP address is designated by the client’s ISP, facilitating the initial TCP/IP connection establishment to the VPN server via the Internet.
Subsequent to the client connecting to the VPN server, a secondary IP address is allocated by the VPN server. This IP address generally belongs to the same subnet as the local network, enabling the client to communicate with the local network.
During VPN server setup, configure a DHCP server to assign addresses to clients, or form a bank of IP addresses for direct client allocation from the VPN server. In either scenario, if the server exhausts valid IP addresses, assigning an address to the client becomes impossible, resulting in connection refusal.
In DHCP server environments, a common configuration error is specifying an erroneous NIC. If the VPN server is right-clicked within the Routing and Remote Access snap-in, and Properties is chosen from the ensuing context menu, the server’s attributes are displayed. The accompanying IP tab has settings allowing the specification of the DHCP source. Ensure that if the DHCP server option is activated, the appropriate network adapter is selected. A network adapter with a TCP/IP path to the DHCP server must be chosen.
2: An illicit connection is permitted.
Subsequently, let’s address the converse issue where unauthorized connections are granted access. Although this situation is less frequent than connectivity issues, it poses increased severity due to the potential security risks and consequent acceptance of unauthorized traffic.
If you review a user’s details in the Active Directory Users and Computers console, the Dial In tab typically includes an option to regulate access via the remote access policy. When this option is activated, and the effective remote access policy allows remote access, the user can link to the VPN.
Though I have not personally encountered this scenario, there are reports of a bug existing in older Windows servers that might permit connection acceptance even if the effective remote access policy bars user connections. As a precaution, especially on legacy server platforms, it is advisable to manage connections directly through the Active Directory Users and Computers console.
Various other security essentials should be in place to hinder unauthorized VPN access. Redundant VPN accounts must be disabled and ideally removed. Users should periodically alter their passwords, and these passwords should meet complexity standards.
All VPN connections should mandate multi-factor authentication, with network firewalls and security services vigilantly monitoring for suspicious or unauthorized connections, triggering urgent alerts when potential issues arise. Implementing these measures enhances the likelihood of averting unauthorized connection acceptance.
3: Unreachable destinations beyond the VPN server.
Another prevalent VPN issue arises when a connection is successfully established, yet the remote user encounters difficulty accessing network resources beyond the VPN server. Predominantly, the most frequent cause is the absence of permission for the user to encompass the entire network.
To grant a user access to the entire network, proceed to the Routing and Remote Access console, right-click on the problematic VPN server. Opt for the Properties command from the ensuing context menu to unveil the server’s attributes, then select the IP tab within the properties sheet. At the apex of the IP tab lies an Enable IP Routing toggle.
If this toggle is activated, VPN users can traverse the entire network provided network firewalls and security services settings allow it. Conversely, if the toggle is inactive, these users can only interact with the VPN server and nothing further.
The issue may also be tied to additional routing concerns. For example, when a user directly connects to the VPN server, it is typically recommended to establish a static route between the client and server.
To create a static route, access the user’s details sheet in Active Directory Users and Computers, navigate to the Dial In tab, and enable the Apply A Static Route option. This prompts Windows to exhibit the Static Routes dialog box. Click on Add Route, include the destination IP address and network mask in the designated area. The metric should remain at 1.
In DHCP server scenarios for assigning IP addresses to clients, several issues could impede users from reaching beyond the VPN server. Duplicate IP addresses are a potential pitfall. If the DHCP server assigns a user an IP address already in use elsewhere on the network, Windows will detect the conflict, and prevent network access for the user.
Another prevalent issue is a lack of address assignment for the user. Typically, if the DHCP server fails to assign an IP address to the user, the connection halts at an earlier stage. Nonetheless, instances exist where address assignment failure occurs, resulting in Windows automatically allotting the user an address from the 169.254.x.x range. If the client receives an address from an absent range within the system’s routing tables, network access will be restricted.
unable to browse the network past the VPN server.
Various other factors may also play a role in this issue. Confirm that the resources the user is trying to reach are indeed within the network they are connected to.
Considering the increasing number of servers, cloud platforms, and software as a service options available, it is plausible that the user might be looking for a resource on an incorrect network or a subnet that the connected network cannot access. It may require a VPN connection to the other subnet. The problem could also lie with a firewall or a security solution, so ensure to assess the settings of these components if they exist between the VPN server and the sought-after resources.
4: A tunnel establishment failure.
If everything seems to be functioning properly but you are unable to establish a tunnel between the client and the server, there are two potential causes for this issue.
The first cause could be the presence of IP packet filters in one or more routers, which might block IP tunnel traffic. It is recommended to check for IP packet filters on the client, server, and any intermediate machines. This can be done by accessing the Advanced button on the TCP/IP Properties sheet of each machine, navigating to the Options tab from the Advanced TCP/IP Settings Properties sheet, selecting TCP/IP Filtering, and clicking on the Properties button.
Another possibility is the existence of a proxy server between the client and the VPN server, performing NAT translation on all traffic between the client and the Internet. This translation might hinder the establishment of a tunnel, particularly if the VPN server anticipates a specific IP address from the client.
It is also important to note that older or less advanced proxy servers (or NAT firewalls) may not support the L2TP, IPSec, or PPTP protocols commonly used in VPN connections.
In some instances, firewall security services or a security solution might obstruct the formation of a VPN tunnel. Check the configurations within these devices or services to ensure that Windows server-based VPN traffic is properly accommodated.
SEE: Securing Linux Policy (TechRepublic Premium)
Additional VPN challenges
Windows server-powered VPNs continue to be a critical solution for securely linking remote users and systems. While specific menus and server properties may evolve over time, the core concepts discussed above often underpin the most prevalent issues. As new server editions, updates, and service packs are rolled out, distinct VPN connection complications and resolutions will emerge.
Fortunately, Microsoft routinely releases updates and guidelines for troubleshooting VPN connections, accessible for monitoring on its website here.
This post was originally published in June 2022. It was revised by Luis Millares in January 2025.
About Author
