Opportunistic Locking and Read Caching on Microsoft Windows Networks

Opportunistic Locking and Read Caching
on Microsoft Windows Networks

A Data Access Worldwide White Paper
by Dennis Piccioni

May 14, 2002
Last Edited: May 17, 2002

Summary

Running applications using DataFlex databases on Windows networks can cause data corruption due to two Windows networking behaviors called opportunistic locking (on Windows servers) and read caching (on Windows clients). This paper discusses these behaviors, their effects and what can be done to minimize the chances of data corruption on Windows networks.

The information in this paper is compiled from the latest information regarding these issues from Microsoft's documentation, our own in-house testing and customer reports. This paper is intended to replace our previous documentation regarding these issues, which included an older white paper (Windows NT Networks Buffering with Windows 95 Clients) and several Knowledge Base articles. Please revisit this white paper from time to time to check for updated information. The Last Edited date at the top of the paper will reflect when the latest edits were made.

Contents

What is Opportunistic Locking?
What is Read Caching?
How Does This Affect Me?
What Operating Systems Are Affected?
Disabling Read Caching on Windows Workstations
Disabling Opportunistic Locking on Windows Servers
Disabling Opportunistic Locking on Windows Workstations
Installing a New Redirector on Windows 95 Workstations
Do Coding Practices Affect These Issues
Persistent Data Corruption
Editing the Windows Registry
Terms
Additional Reading

What is Opportunistic Locking?

Opportunistic locking is a Windows-specific mechanism for client/server databases to allow multiple processes to lock the same file while allowing for local (client) data caching to improve performance over Windows networks.

Microsoft's documentation states "An opportunistic lock (also called an oplock) is a lock placed by a client on a file residing on a server. In most cases, a client requests an opportunistic lock so it can cache data locally, thus reducing network traffic and improving apparent response time. Opportunistic locks are used by network redirectors on clients with remote servers, as well as by client applications on local servers" and "Opportunistic locks are requests from the client to the server. From the point of view of the client, they are opportunistic. In other words, the server grants such locks whenever other factors make the locks possible.".

You can read more about opportunistic locking in Microsoft's documentation. Please see the Additional Reading section for more information.

What is Read Caching?

Read caching is local caching of read data, sometimes referred to as read-ahead caching, on client (workstation) PCs. This type of caching is the default behavior of the Windows operating system to improve performance over Windows networks. The way it works is that workstations can read data from the server and keep a local copy of that data to reduce network traffic. As soon as that workstation writes any data, it should flush file buffers at that time, reread the data from the server and then commit the changes.

Problems with read caching usually occur if something unforeseen happens, such as a workstation crash, where data is not properly flushed from the workstation, which can lead to data corruption.

Microsoft's documentation states that "Under extreme conditions, some multiuser database applications that use a common data store over a network connection on a file server may experience transactional integrity issues or corruption of the database files and/or indexes stored on the server. This typically applies to some so-called "ISAM style", or "record oriented" multiuser database applications, not to a client/server relational system like SQL Server." and "A hazard of local caching is that written data only has as much integrity as the client itself for as long as the data is cached on the client. In general, locally cached data should be flushed to the server as soon as possible."

You can read more about read caching in Microsoft's documentation. Please see the Additional Reading section for more information.

How Does This Affect Me?

The DataFlex database is an ISAM database and thus more likely to be adversely affected by read caching and opportunistic locking than client/server databases. Our best recommendation is to run your existing VDF and DataFlex applications with data files (tables) converted to a client/server database, such as Pervasive.SQL, IBM DB2 or Microsoft SQL.

Data Access Worldwide has direct drivers (Connectivity Kits) available for Pervasive.SQL, IBM DB2 and Microsoft SQL, as well as an ODBC Connectivity Kit for access to any ODBC-compliant database. All of these drivers are loaded at runtime and require no coding changes to be used with existing VDF, DataFlex or WebApp Server applications.

If you wish to continue using the DataFlex database, you can use the information in this paper to allow Windows networks to work properly with the DataFlex database by disabling opportunistic locking and read caching.

Disabling opportunistic locking and read caching will have a performance impact on any database application, whether an ISAM database or client/server database is used, but it will not affect the data integrity of any database.

What Operating Systems are Affected?

All Windows operating systems in the NT family that act as database servers for DataFlex data files (meaning that DataFlex data files are stored there and accessed by other Windows PCs) need to have opportunistic locking disabled in order to minimize the chances of database corruption. This includes Windows NT, Windows 2000 and Windows XP.

All Windows operating systems in the 9X family that act as database clients for DataFlex data files (meaning that they access DataFlex data files stored on other Windows PCs) need to have read caching disabled in order to minimize the chances of database corruption. This includes Windows 95, Windows 98 and Windows ME.

Disabling Read Caching on Windows Workstations

The Windows registry entry that controls read caching on Windows network clients is:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VxD\VREDIR

DiscardCacheOnOpen REG_BINARY 0 or 1
Default: 0 (not disabled)

To disable read caching, the value of DiscardCacheOnOpen must be set to 1.

You can verify (or change or add, if necessary) this Registry value using the Windows Registry Editor (regedit.exe).
If you do change this Registry value, you will have to reboot the PC to ensure that the new setting goes into effect.

Please read the Microsoft disclaimer regarding editing of the Windows registry here.

STEPS:

1. Start > Run > Regedit.exe
2. Click on the + (plus sign) next to HKey_Local_Machine
3. Click on the + (plus sign) next to System
4. Click on the + (plus sign) next to CurrentControlSet
5. Click on the + (plus sign) next to Services
6. Click on the + (plus sign) next to VxD
7. Click on the VREDIR entry on the left-hand side of Registry Editor

8. If the DiscardCacheOnOpen registry value already exists (on the right-hand side of Registry Editor), ensure that its value is 1

9. If the DiscardCacheOnOpen value already exists but its value is not 1, double-click on DiscardCacheOnOpen to change its value to 1

10. If the DiscardCacheOnOpen entry does not exist, right-click in the white space of the right-hand side of Registry Editor
11. Select New > Binary value
12. Rename the value to DiscardCacheOnOpen
13. Double-click on DiscardCacheOnOpen to change its value to 1

Disabling Opportunistic Locking on Windows Servers

There are 2 Windows registry entries that control opportunistic locking (oplocks) on Windows network servers:

1. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters EnableOpLockForceClose
2. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters EnableOplocks

1. EnableOpLockForceClose

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters

EnableOpLockForceClose REG_DWORD 0 or 1
Default: 0 (not disabled)

To disable oplocks, the value of EnableOpLockForceClose must be set to 1.

You can verify (or change or add, if necessary) this Registry value using the Windows Registry Editor (regedit.exe).
If you do change this Registry value, you will have to reboot the PC to ensure that the new setting goes into effect.

Please read the Microsoft disclaimer regarding editing of the Windows registry here.

STEPS:

1. Start > Run > Regedit.exe
2. Click on the + (plus sign) next to HKey_Local_Machine
3. Click on the + (plus sign) next to System
4. Click on the + (plus sign) next to CurrentControlSet
5. Click on the + (plus sign) next to Services
6. Click on the + (plus sign) next to LanManServer
7. Click on the Parameters entry on the left-hand side of Registry Editor

8. If the EnableOpLockForceClose registry value already exists (on the right-hand side of Registry Editor), ensure that its value is 1

9. If the EnableOpLockForceClose value already exists but its value is not 1, double-click on EnableOpLockForceClose to change its value to 1

10. If the EnableOpLockForceClose entry does not exist, right-click in the white space of the right-hand side of Registry Editor
11. Select New > DWORD value
12. Rename the value to EnableOpLockForceClose
13. Double-click on EnableOpLockForceClose to change its value to 1

2. EnableOplocks

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters

EnableOplocks REG_DWORD 0 or 1
Default: 1 (true)

To disable oplocks, the value of EnableOplocks must be set to 0.

You can verify (or change or add, if necessary) this Registry value using the Windows Registry Editor (regedit.exe).
If you do change this Registry value, you will have to reboot the PC to ensure that the new setting goes into effect.

Please read the Microsoft disclaimer regarding editing of the Windows registry here.

STEPS:

1. Start > Run > Regedit.exe
2. Click on the + (plus sign) next to HKey_Local_Machine
3. Click on the + (plus sign) next to System
4. Click on the + (plus sign) next to CurrentControlSet
5. Click on the + (plus sign) next to Services
6. Click on the + (plus sign) next to LanManServer
7. Click on the Parameters entry on the left-hand side of Registry Editor

8. If the EnableOplocks registry value already exists (on the right-hand side of Registry Editor), ensure that its value is 0

9. If the EnableOplocks value already exists but its value is not 0, double-click on EnableOplocks to change its value to 0

10. If the EnableOplocks entry does not exist, right-click in the white space of the right-hand side of Registry Editor
11. Select New > DWORD value
12. Rename the value to EnableOplocks
13. Double-click on EnableOplocks to change its value to 0

Note: The location of the client registry entry for opportunistic locking has changed in Windows 2000 from the earlier location in Microsoft Windows NT. In Windows 2000, the registry entry that disables opportunistic locking is:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MRXSmb\Parameters\

OplocksDisabled REG_DWORD 0 or 1
Default: 0 (not disabled)

To disable oplocks, the value of OplocksDisabled must be set to 1.

Note: Windows 2000 will still respect the EnableOplocks registry value used to disable oplocks in earlier versions of Windows.

Disabling Opportunistic Locking on Windows Workstations

If you are using a Windows NT family workstation in place of a server, you must also disable opportunistic locking (oplocks) on that workstation. For example, if you use a PC with the Windows NT Workstation operating system instead of Windows NT Server, and you have DataFlex data files located on it that are accessed from other Windows PCs, you will need to disable oplocks on that system.

The major difference is the location in the Windows registry where the values for disabling oplocks are entered. Instead of the LanManServer location, the LanManWorkstation location is used here.

There are 2 Windows registry entries that control opportunistic locking (oplocks) on Windows network workstations:

1. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters EnableOpLockForceClose
2. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters EnableOplocks

1. EnableOpLockForceClose

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters

EnableOpLockForceClose REG_DWORD 0 or 1
Default: 0 (not disabled)

To disable oplocks, the value of EnableOpLockForceClose must be set to 1.

You can verify (or change or add, if necessary) this Registry value using the Windows Registry Editor (regedit.exe).
If you do change this Registry value, you will have to reboot the PC to ensure that the new setting goes into effect.

Please read the Microsoft disclaimer regarding editing of the Windows registry here.

STEPS:

1. Start > Run > Regedit.exe
2. Click on the + (plus sign) next to HKey_Local_Machine
3. Click on the + (plus sign) next to System
4. Click on the + (plus sign) next to CurrentControlSet
5. Click on the + (plus sign) next to Services
6. Click on the + (plus sign) next to LanManWorkstation
7. Click on the Parameters entry on the left-hand side of Registry Editor

8. If the EnableOpLockForceClose registry value already exists (on the right-hand side of Registry Editor), ensure that its value is 1

9. If the EnableOpLockForceClose value already exists but its value is not 1, double-click on EnableOpLockForceClose to change its value to 1

10. If the EnableOpLockForceClose entry does not exist, right-click in the white space of the right-hand side of Registry Editor
11. Select New > DWORD value
12. Rename the value to EnableOpLockForceClose
13. Double-click on EnableOpLockForceClose to change its value to 1

2. EnableOplocks

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanManWorkstation\Parameters

EnableOplocks REG_DWORD 0 or 1
Default: 1 (true)

To disable oplocks, the value of EnableOplocks must be set to 0.

You can verify (or change or add, if necessary) this Registry value using the Windows Registry Editor (regedit.exe).
If you do change this Registry value, you will have to reboot the PC to ensure that the new setting goes into effect.

Please read the Microsoft disclaimer regarding editing of the Windows registry here.

STEPS:

1. Start > Run > Regedit.exe
2. Click on the + (plus sign) next to HKey_Local_Machine
3. Click on the + (plus sign) next to System
4. Click on the + (plus sign) next to CurrentControlSet
5. Click on the + (plus sign) next to Services
6. Click on the + (plus sign) next to LanManWorkstation
7. Click on the Parameters entry on the left-hand side of Registry Editor

8. If the EnableOplocks registry value already exists (on the right-hand side of Registry Editor), ensure that its value is 0

9. If the EnableOplocks value already exists but its value is not 0, double-click on EnableOplocks to change its value to 0

10. If the EnableOplocks entry does not exist, right-click in the white space of the right-hand side of Registry Editor
11. Select New > DWORD value
12. Rename the value to EnableOplocks
13. Double-click on EnableOplocks to change its value to 0

Installing a New Redirector on Windows 95 Workstations

On versions of Windows 95 prior to the OSR2 release, in addition to disabling read caching by making the appropriate Windows registry settings, you may have to install a new redirector (vredir.vxd) as well. The OSR2 version of Windows 95, comes pre-installed with the newer redirector.

Microsoft has provided a new vredir.vxd for Windows 95 (version 4.00.955) that does disable the buffering of shared files on Windows networks. If you have a version of the redirector prior to 4.00.955, you should install the new redirector. The redirector executable is a self-installing program provided by Microsoft. Run it under Windows 95 and it installs itself and makes the needed changes to your Windows registry. You then need to restart Windows 95 for the changes to take effect.

You can download the updated redirector from our FTP site: ftp://ftp.dataaccess.com/pub/products/dataflex/misc/vrdr2upd.exe.

This new redirector removes the local caching that the old redirector included. The local caching would cause data errors when retrieving data from a Windows server. The new redirector has been extensively tested by Data Access Corporation and does indeed solve the problems with data reliability between a Windows 95 workstation and a Windows server. This redirector is not needed if accessing data on a Novell NetWare server. However, installing the new redirector will have no detrimental effects on sites not using Windows servers. Thus, our general recommendation would be to use the new redirector at all customer sites.

Note: When you search the Windows registry for vredir.vxd on a Windows 95 machine that has not had either the patch or the OSR2 revision installed, the first entry found is simply the component name and its associated data (displayed in hex). On any machine which was shipped with the OSR2 release, or had the patch installed, the first entry that is found displays a comment about "Samba UNIX" and the revision number 4.00.955 beneath the comment (the screen with the hex data will be the second one found).

Do Coding Practices Affect These Issues?

If your code is using DataDictionaries and/or Data_Sets, there should be no additional problems after opportunistic locking and read caching have been disabled.

If you have any code that is not using DataDictionaries and/or Data_Sets, for example, in a find loop using the record buffer for finding, we have seen some instances where, even with opportunistic locking and read caching disabled, that data in records that are new or were edited since the data was first accessed is not retrieved properly. Workarounds are doing the find in a locked state or issuing a reread command after each record is found (remember to issue an unlock command after the reread as a reread performs a lock as part of its functionality). We are still researching how to get around this Microsoft operating system problem. We will publish any additional information we obtain as soon as possible.

We have tried using the Win32 FlushFileBuffers API call that Microsoft recommends in their documentation in our runtime when the DF_HIGH_DATA_INTEGRITY attribute was turned on. However, the applications' performance degraded to the point that it was virtually unusable when doing so, because this API call is a very generic call that flushes all buffers on a workstation instead of just those used by one application.

Persistent Data Corruption

If you have applied all of the settings discussed in this paper but data corruption problems and other symptoms persist, here are some additional things to check out:

We have credible reports from developers that faulty network hardware, such as a single faulty network card, can cause symptoms similar to read caching and data corruption.
If you see persistent data corruption even after repeated reindexing, you may have to rebuild the data files in question. This involves creating a new data file with the same definition as the file to be rebuilt and transferring the data from the old file to the new one. There are several known methods for doing this that can be found in our Knowledge Base.

Editing the Windows Registry

Caution: The following warning appears in every Microsoft Knowledge Base article that discusses editing the Windows registry:

WARNING : Using Registry Editor incorrectly can cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your own risk.

For information about how to edit the registry, view the "Changing Keys and Values" Help topic in Registry Editor (Regedit.exe) or the "Add and Delete Information in the Registry" and "Edit Registry Data" Help topics in Regedt32.exe. Note that you should back up the registry before you edit it. If you are running Windows NT or Windows 2000, you should also update your Emergency Repair Disk (ERD).

Terms

ISAM:
Indexed Sequential Access Method is a file management system developed at IBM that allows records to be accessed either sequentially (in the order they were entered) or randomly (with an index).

Additional Reading

You may want to check for an updated version of this white paper on our Web site from time to time. Many of our white papers are updated as information changes. For those papers, the Last Edited date is always at the top of the paper.

Section of the Microsoft MSDN Library on opportunistic locking:
Opportunistic Locks, Microsoft Developer Network (MSDN), Windows Development > Windows Base Services > Files and I/O > SDK Documentation > File Storage > File Systems > About File Systems > Opportunistic Locks, Microsoft Corporation.
URL=http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fsys_5yk3.asp

Microsoft Knowledge Base Article Q224992 "Maintaining Transactional Integrity with OPLOCKS", Microsoft Corporation, April 1999, URL=http://support.microsoft.com/default.aspx?scid=kb;en-us;Q224992

Microsoft Knowledge Base Article Q296264 "Configuring Opportunistic Locking in Windows 2000", Microsoft Corporation, April 2001, URL=http://support.microsoft.com/default.aspx?scid=kb;en-us;Q296264

Microsoft Knowledge Base Article Q129202 "PC Ext: Explanation of Opportunistic Locking on Windows NT", Microsoft Corporation, April 1995, URL=http://support.microsoft.com/default.aspx?scid=kb;en-us;Q129202

Visit the Data Access Worldwide Support Home page for information about all of our support offerings, including free support offerings, such as the Technical Knowledge Base, white papers and Peer Support Newsgroups: http://www.dataaccess.com/support.

DAW Knowledge Base:
Visit the Data Access Worldwide Knowledge Base, a great resource for the latest technical information about all Data Access Worldwide products: http://www.dataaccess.com/kbase.asp.

DAW Newsgroups:
Visit the Data Access Worldwide Developer Community Newsgroups for sharing information about Data Access Worldwide products with other developers and users: http://www.dataaccess.com/content.asp?PageCode=SUP_C.

Contacting Data Access Worldwide

Data Access Worldwide
14000 SW 119 Ave
Miami, FL 33186
305-238-0012
Domestic Sales: 800-451-3539
Fax: 305-238-0017
email: sales@dataaccess.com
Newsgroup Server: news.dataaccess.com
Internet: http://www.dataaccess.com

Data Access Technical Support
800-451-3539 / 305-232-3142
email: support@dataaccess.com
Visit our Support Home page to see all of our Support options: http://www.dataaccess.com/support

Copyright Notice
This document is property of Data Access Corporation. With credit to Data Access Corporation for its authorship, you are encouraged to reproduce this information in any format either on paper or electronically, in whole or in part. You may publish this paper as a stand alone document within your own publications conditional on the maintenance of the intent, context, and integrity of the material as it is presented here.

DataFlex is a registered trademark of Data Access Corporation.
Windows is a registered trademark of Microsoft Corporation.

NO LIABILITY FOR CONSEQUENTIAL DAMAGES
To the maximum extent permitted by applicable law, in no event shall Data Access Corporation be liable for any special, incidental, indirect, or consequential damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or any other pecuniary loss) arising out of the use of or inability to use any information provided in this document, even if Data Access Corporation has been advised of the possibility of such damages. Because some states and jurisdictions do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you.