Using DLL's with DataFlex

An AE Group White Paper

Application Engineering Group Data Access Corporation
Publication date: April 25, 1996

 

Overview

This document extends existing DataFlex documentation and provides basic information and suggestions on DLLs. The discussion
is from the point of view of an experienced C/C++ and DataFlex programmer. DLL issues, however, cover a broad and advanced
range, so this discussion is far from exhaustive.

Data Access' Product Services Department policy provides only limited technical support on DLL use. Technical support covers only
the use of DataFlex commands and examples that interface with DLLs, leaving responsibility for the DLL's creation and functioning to
the developer.

This information is subject to change without notice, as the products involved and our understanding of their interrelationship changes.

Background

A Dynamic-Link Library (DLL) is a library of functions dynamically linked to an executable during runtime, as opposed to a static
library, where the functions are linked during compilation. The library's being dynamic provides a couple of advantages:

  • The DLL may be developed with a different compiler, language, and/or development system from that of the user of
    the DLL (DataFlex in this case).
  • The DLL is loaded only once, regardless of the number of programs linked with it. This reduces memory usage and load
    times.

Of course, there are disadvantages also:

  • Error checking is generally weaker than when linking statically.
  • The implementation of the DLL's loading, unloading, and function calling is platform-specific.

When DataFlex for Windows was created, being able to make DLL calls was a natural feature opening new possibilities for DataFlex
developers.

  • Windows features and other products can now be accessed through their DLLs.
  • The developer who needs to extend the language can create a DLL instead of editing the runtime with DataFlex's C Source
    and Library. In this way, the extensions are independent of the version of DataFlex and can be added with the developer's
    choice of tools and compiler.

Because most of our inquiries on DLLs have been about calling other DLLs from DataFlex, this paper focuses on that. Any discussion about creating DLLs to extend DataFlex is reserved for another paper.

The use of DLLs depends largely on the operating system. While many operating systems support the use of DLLs, Windows is
currently the only platform where DataFlex supports them; thus, this discussion assumes the environment and DataFlex platform is
Windows in all cases.

Furthermore, while a DLL can theoretically be produced in any language that creates executable images, C/C++ is, by far, the
language most-commonly used. So, this discussion also assumes the DLL was created in C/C++.

Memory Models

A Windows DLL is built under one of two memory models, 16-bit or 32-bit, just like an executable. This will affect:

  • Which version of Windows under which the DLL will work
Windows Version: 16-Bit DLLs 32-Bit DLLs
Windows 3.1/3.11 Yes No
Win32s Yes Yes
Windows95 Yes* Yes
WindowsNT No Yes

*In Windows95, the executable must perform a special action called 'thunking' in order to access 16-bit DLLs. Many products able to
access 16-bit DLLs in Win32s are not able to do so in Windows 95.

  • Which version of DataFlex with which the DLL will work
DataFlex Version: 16-Bit DLLs 32-Bit DLLs
3.01b Yes No
3.05d w/ Win32s Yes Yes
3.05d w/ Windows 95 or NT No Yes
3.1c Console Mode No Yes
4.0 Visual DataFlex No Yes
  • How functions are 'imported' into DataFlex.
  • How pointers are defined in C/C++.

Importing a DLL Function in DataFlex

Importing a function from a DLL to any program involves:

  1. Telling the program the name and location of the DLL.
  2. Telling the program the name of the function as the DLL knows it.
  3. Telling the program what parameters the function expects and will return.

All three steps must be done before the function can be used. In other words, the function must be 'imported'. In DataFlex, this is
done with the commands external_function, external_function16, or external_function32.

Consider the example:

external_function32 Message_Beep "MessageBeep" user32.dll ;
    word iSound returns integer

  1. The 'import' name of the function is MessageBeep. This term is case-sensitive and must be exactly as the DLL knows it. In
    16-bit DLLs, the import name is always uppercase.
  2. The name of the DLL is user32.dll. Since a full pathname was not given (e.g., c:\windows\system\user32.dll), DataFlex will ask
    Windows for the location of the DLL (see 'How DataFlex Searches for and Loads a DLL').
  3. MessageBeep expects a "word" (one of several types of integers supported by the C language) and returns an integer.
    The word itself is represented by iSound , which describes what the parameter is used for.

Notes:

  • As versions of software have changed, DLLs of the same name have changed. The DLLs, such as 'shell.dll' of
    Windows95 and Win32s, have the same name and can act very differently. It is possible, especially after upgrading, to
    have DLLs from different revisions of software on your machine. Be sure you are specifying the correct DLL when
    you import.
  • The 'import' name as the DLL knows it may not be the same as the user or even the developer (see "Mangled Names in
    C/C++" and "Actual Function Name vs. Documented") DataFlex data types are not the same as C/C++ data types,
    so a DataFlex Integer is not the same as a C Integer (see "Passing Integers to DLL Functions").
  • When C/C++ functions have no return value, the return type is 'void'. If this is the case, use return type integer with
    external_function. The function should always return 0. There is an external_procedure, but it does not work and is
    not expected to be continued in future revisions.

If a mistake has been made in any aspect of 'importing,' you may get an exception error, DataFlex error, or no error at all. The results are unpredictable, so thoroughly test all code that relies on DLL calls.

When to Use External_Function, External_Function16, and External_Function32

Which command you should use will depend on the memory model of the DLL. External_function16 can be used if the DLL is 16-bit,
and external_function32 can be used if the DLL is 32-bit. However, DataFlex 3.01b only supports external_function and
16-bit DLLs. After 3.01b, external_function imports the default memory model of 32-bit. The default model type can be changed
using set_default_call_type.

16-bit DLL 32-bit DLL
DataFlex for Windows 3.01b Use external_function Not Supported
DataFlex for Windows later than 3.01b Use external_function16 or external_function if set_default_call_type 16 Use external_function32 or external_function

There really is no reason to use external_function or set_default_call_type, and they are just confusing in the end. They
exist for backward compatibility.

How DataFlex Searches for and Loads a DLL

DataFlex will search for the DLL file in the same way that all Windows executables do:

  • The directory in which DFRUN.EXE resides.
  • The current directory.
  • The Windows system directory, usually c:\windows\system.
  • The Windows directory, usually c:\windows.
  • The directories in the system PATH environment variable.

Note: Your current directory is not the first one searched. Also, remember the current directory in Windows depends on what
method was used to start the application: from an icon, from the Program Manager, from the prompt, etc.

DLLs are loaded only once. If another process makes a call to a DLL already loaded, a reference counter is incremented. When the
process terminates, the reference count decrements. The DLL does not unload until all processes using the DLL have terminated.

C/C++

While a full discussion of DLL development is beyond the scope of this document, some discussion will provide useful insight. If you wish to know more, most books covering the topic of writing Windows programs in C/C++ include full discussions on how to create DLLs.

Exporting DLL Functions in C/C++:

'Exporting' a function makes 'importing' possible for other programs. In other words, the function's name, parameters, return type, and
calling methods are stored in the DLL so that they are accessible from outside the DLL. There are three methods for 'exporting'
functions in C/C++:

  1. The 'exported' function can be listed as such in a Windows-supported ASCII file called a .DEF file (not to be
    confused with the .DEF files used by DataFlex to generate datafiles).
  2. The function declaration and definition may include syntax telling the compiler/linker to 'export' the function. The syntax
    depends on the C compiler, e.g., Borland or Microsoft.
  3. Some compilers include an option that 'exports' all functions in the DLL.

Each of these is fully discussed in the compiler's documentation and many C/C++ tutorials.

Mangled Names in C/C++:

A C++ function can be 'overloaded'-that is, two functions with the same name but different parameters. To ensure uniqueness, a
compiler may 'mangle' the function name. 'Mangling' involves changing the internal representation of the function name based on
such aspects as parameters and return type. Thus, the name used by the program will be different from the one you specified. The
problem is that when the function gets 'exported', the 'mangled' name is the 'exported' name in the DLL.

If the DLL user is using the same compiler as the DLL developer, he can use a .LIB file that translates the C++ function name to the
'mangled' names. However, if the DLL user is using another compiler or language such as DataFlex, the 'mangled' name must be
'imported'. Since each compiler can 'mangle' automatically and differently, the DLL developer can be ignorant of what the 'mangled'
name is or even if the name has been 'mangled'.

Fortunately, most C compilers for Windows come with a utility, 'impdef', that creates a Windows .DEF file from the DLL that will
contain the proper name to be 'imported'. Another such utility is 'dumpbin', which outputs the 'exported' names from the DLL. If you
suspect the function name has been 'mangled', use these utilities to ensure that the correct name is being used.

Not 'mangling' the function name usually involves telling the compiler to adhere strictly to the 'C' language instead of 'C++'. This is often done with the declaration ' extern "C" ', but the exact method depends upon the C compiler. If you are lucky enough to have the header files, you can check for this; otherwise, try to find out if the developer used 'C' or 'C++'. If you cannot confirm that he used 'C',
the names may have been 'mangled'.

Actual Function Names vs. Documented:

Many DLL functions have names other than the one described in their documentation. An example of this is the Windows API
function, MessageBox. The actual name, as the DLL knows it, is MessageBox if using the 16-bit DLL, but MessageBoxA if using the
32-bit DLL. When you compile in C/C++, the name MessageBox is translated to the appropriate form from header files. But this
information is not provided by the DLL, so DataFlex relies on the developer to provide the actual name. In other words, the only way
to be certain of the correct DLL function name is to check the header file. If a DLL call is not successful, check the header file,
especially if the function name matches that in the DLL's documentation. You can also use utilities such as 'impdef' and
'dumpbin' to output the correct names from DLL directly.

C/C++ Calling Conventions:

In many DOS and Windows C/C++ compilers, the developer can control how arguments are passed at the assembly level. This is
usually more low-level and detailed than even most C/C++ programmers wish to go, so suffice it to say for this discussion that
there are several calling conventions. Only the standard calling conventions for Windows C/C++ (pascal in 16-bit, _stdcall
otherwise) are supported in DataFlex.

Calling DLL Functions from DataFlex

Once the functions have been 'imported', the function is now accessible as a global DataFlex function.

Here is an example of this under Windows95:

use dll

external_function32 Message_Beep "MessageBeep" user32.dll ;
    integer iSound returns integer

integer iThrowAway
move (Message_Beep(-1)) to iThrowAway

Message_Beep will create a short computer sound depending on the integer value passed in iSound.

If the user does not have a sound card or driver, '-1' is the only acceptable value. Message_Beep will return a 1 if successful and 0
otherwise, but in our example, we do not make use of the return value. The example would be the same if Message_Beep did not
return anything ('void').

So far, a DLL function looks like a standard DataFlex global function, but there is one important difference: DLL functions need
arguments passed that can be converted in C/C++.

Passing Integers to DLL Functions:

Integer arguments are the simplest because the concept of an 'integer' is the same in DataFlex and C/C++. The only difference is that in DataFlex an integer is of only one size while in C/C++ on PCs, an integer can be from 1 byte (short) to 4 bytes (long). Exactly what size an integer will be depends upon the memory model the DLL was compiled with. To help with this, Windows has defined the types 'BYTE', 'WORD', and 'DWORD' for 1, 2, and 4 bytes respectively; but not all developers define their functions with these. The sizes of other integer types are:

  • 'char' is 1 byte
  • 'short' is 2 bytes
  • 'int' is 2 bytes in 16-bit and 4 bytes in 32-bit
  • 'long' is 4 bytes

For DataFlex, the size of the integer passed makes no difference, so if the parameter is char, short, int, long, BYTE, WORD, or
DWORD, use integer.

To be more compatible, three new types were added to DataFlex. The DWORD data type was added and is equivalent to a DataFlex
integer. BYTE and WORD was also added, but only as structure field types. They have no meaning outside of a structure, and you will get errors if you try to use them as varibles. (see Passing Structures to DLL Functions).

Passing Strings to DLL Functions:

C/C++ does not have a string data type as DataFlex does. Instead, a pointer or address is passed from an array of 1-byte integers ('char'), which then allows the C/C++ function to read and modify the value. The steps to get the pointer/address of a DataFlex String are as follows:

  1. Initialize the DataFlex string and make sure DataFlex allocates enough space for it.
  2. Use the getaddress command to pass the string's address to DataFlex type POINTER.

Here is an example of getting the address of a global string.

external_function MyDLLStringFunc "MyStringFunc" examp.dll ;
    pointer String_Ptr returns integer

string My_Str 255
pointer My_Ptr
integer iThrowAway

move (repeat( character(0), 255 )) to My_Str
getaddress of My_Str to My_Ptr

move (MyDLLStringFunc(My_Ptr)) to iThrowAway

In the above example, MyStringFunc actually returns type 'void', so the integer iThrowAway will always be 0. The real results are placed in My_Str, which was modified using My_Ptr. Of course, MyStringFunc could also receive string values as well as return them. In fact, the only way to receive a string from a DLL call is to pass it the string pointer.

In the case of global strings, DataFlex will automatically allocate the size of the string in bytes, 255 in our example. However, initializing the string with char(0) is still a good idea because C/C++ uses the ASCII value 0 to designate the end of the string. These char(0) can also be present on strings returned from C/C++, so DataFlex added the function cstring, which returns the string with all the char(0) removed.

Whether the string's address is obtained before or after it is initialized is unimportant with global strings, but critical with local strings. Here is an example:

external_function MyDLLStringFunc "MyStringFunc" examp.dll ;
    pointer String_Ptr ;
    returns integer

function My_String_Func returns string
    local string My_Str
    local pointer My_Ptr
    local integer iThrowAway

    move (repeat( character(0), 255 )) to My_Str
    getaddress of My_Str to My_Ptr

    move (MyDLLStringFunc(My_Ptr)) to iThrowAway

    function_return (cstring(My_Str))
end_function

string Global_Str 255
get My_String_Func to Global_Str

In the case of local strings, memory is not allocated until it is initialized. In other words, the string does not exist until it has been
given a value, and it only has enough memory to contain the value you have given it. The following is bound to cause errors:

local string My_Str
local pointer My_Ptr
local integer iThrowAway

getaddress of My_Str to My_Ptr
move (MyDLLStringFunc(My_Ptr)) to iThrowAway

Even though I got the pointer correctly from My_Str, there was no string from which to get a pointer. The following will also not work
because My_Str does not come into existence until after I tried to get its pointer.

getaddress of My_Str to My_Ptr
move (repeat( character(0), 255 )) to My_Str
move (MyDLLStringFunc(My_Ptr)) to iThrowAway

Also, a DataFlex string can be re-allocated if its value changes. Consider this example:

move (repeat( character(0), 255 )) to My_Str
getaddress of My_Str to My_Ptr
move (MyDLLStringFunc(My_Ptr)) to iThrowAway

move (repeat( character(0), 255 )) to My_Str
move (MyDLLStringFunc(My_Ptr)) to iThrowAway

The first call to MyDLLStringFunc should have no problems, but the second may not work because My_Ptr may have become invalid
when My_Str was re-initialized. Instead:

move (repeat( character(0), 255 )) to My_Str
getaddress of My_Str to My_Ptr
move (MyDLLStringFunc(My_Ptr)) to iThrowAway

move (repeat( character(0), 255 )) to My_Str
getaddress of My_Str to My_Ptr
move (MyDLLStringFunc(My_Ptr)) to iThrowAway

As a rule, you should get the address of all strings immediately prior to calling the DLL function.

Passing Structures to DLL Functions:

C/C++ allows different data types to be combined in records or structures. As with strings, a pointer to the structure is usually
passed. In the case of DataFlex structures (added in the DLL.PKG), the structure data is stored in a string, so passing a pointer to a
structure follows the same rules as passing a pointer to a string. Here is an example.

external_function MyDLLStuctFunc "MyStructFunc" examp.dll ;
    pointer Struct_Ptr returns integer

type My_Struct_Type
    field My_Struct_Type.Integer0 as dword
    field My_Struct_Type.Integer1 as dword
end_type

function My_Struct_Func returns string
    local string My_Struct
    local pointer My_Ptr
    local integer iThrowAway

    zerotype My_Struct_Type to My_Struct
    getaddress of My_Struct to My_Ptr

    move (MyDLLStuctFunc(My_Ptr)) to iThrowAway


    function_return My_Struct
end_function

string Global_Struct
get My_String_Func to Global_Struct

integer Int0 Int1

getbuff from Global_Struct at My_Struct_Type.Integer0 to Int0
getbuff from Global_Struct at My_Struct_Type.Integer1 to Int1

The creation and use of structures is covered in the DataFlex Windows documentation, so not much more than this example is
needed. There are, however, a couple of points I would like to elaborate. First, zerotype... was used instead of move (repeat(... to
initialize the structure. Move (repeat(... could have been used just as easily, but zerotype removes the need to know the structure's size. Second, when getbuff and put accessed the string to read and write data, they used the size of each field of the structure to determine at what offset to read/write. C/C++ will do the same, so it is critical that the sizes of the DataFlex-structure fields match exactly with the C/C++ fields. For example, if the C/C++ structure were:

typedef struct Tag_My_Struct_Type {
    int Int0 ;
    int Int1 ;
} My_Struct_Type

My_Struct_Type.Int0 and My_Struct_Type.Int1 are 2 bytes in a 16-bit DLL and 4 bytes in a 32-bit DLL while the DataFlex field
My_Struct_Type.Int0 and My_Struct_Type.Int1 will always be 4 bytes. Therefore, the DataFlex program would pass the wrong structure if the DLL were 16-bit. In some cases, the C/C++ developer will use types like 'WORD' and 'DWORD', which have the same size in all memory models; otherwise, you will have to know the memory model to know the correct field size. In either case, you should use DataFlex types such as WORD and DWORD to avoid any confusion. Remember that a C/C++ 'int' is not necessarily the same size as a DataFlex Integer.

In Summary:

When defining structures, make sure that the field sizes match exactly with those defined in C++. C++ structure 'int' fields
(such as BYTE, WORD, DWORD, char, short, int, long, etc.) should be defined as BYTE, WORD, or DWORD in
DataFlex. DWORD and integer should be used for passing C++ integers. BYTE and WORD cannot be used.
Strings and structures are passed by passing their address. The variable should be initialized and a new address obtained
before each call to the DLL.

Improvements in DataFlex for Windows after 3.05d

Some commands and data types added are:

The integer type byte is added. The command zerostring is added to initialize strings instead
of move (repeat(character(0),…

ZEROSTRING String_Size TO Buffer

Fills string Buffer with String_Size number of ASCII 0's.

Structures are extended to include String fields and commands were added to access them.

PUT_STRING Value TO Destination AT Field

Overwrites a field of a destination structure with value. If Value is too small, the field will be padded with ASCII 0's. Otherwise, if
Value is too large, it will be truncated to fit the field.

GETBUFF_STRING FROM Buff AT Field TO Value

Returns a string of bytes, value, from a structure, buff. The length of the string will be that of the field and will include values such as
ASCII 0's.

Other commands added to increase usability.

OFFSET_OF_FIELD Field_Address TO Value

Returns the string position of Field_Address to Value.

SIZE_OF_FIELD Field_Address TO Value

Returns the byte size of Field_Address to Value.

Limitations of Using DLLs within DataFlex

Even though DataFlex supports calls to DLL function, there may still be limitations due to features not available in the DataFlex language. For example, some DLL calls take pointers to functions as parameters. Since, you cannot obtain the pointer to a DataFlex function, DataFlex functions cannot be passed in DLL calls. Obviously, if the function was defined outside of DataFlex, and there
existed a DLL call to obtain its pointer, then that function could be passed.

Another example is the fact that you cannot obtain the handle of a DataFlex object while using the Console Mode runtime. Many
Windows API calls require the handle a visual object to function. To accommodate this, the Windows version of DataFlex provides a
method, Container_Handle. However, the method Container_Handle is not available in the Console Mode runtime, so
any Windows API call requiring an object handle cannot be used.

Error Checking While Using DLLs

DataFlex is unable to report most errors that occur while using DLLs, so most of the areas I have covered will go unnoticed if you
do them incorrectly-the program will just seem not to work. This is perhaps the most-difficult aspect of using DLLs in DataFlex.
Unfortunately, there are very few standards in C/C++ concerning error reporting, and even fewer apply to DLLs. For the most part,
you will just have to be careful.

When the DLL function does not seem to work, there are three possibilities:

  1. The DLL function is called properly but is not working at the C/C++ level. In this case, you are limited to whatever
    debugging the DLL provides unless you have written the DLL yourself.
  2. The DLL function is working and being called properly. Instead, the error is somewhere else in your DataFlex code.
  3. The DLL function is working but not being called properly.

In the first case, if you wrote the DLL, of course you will be able to use C/C++ debug tools and techniques to catch and report;
otherwise, you will have to rely on whatever documentation the developer provided.

One of the few conventions of C/C++ error reporting is to have the function return an error code. If you are fortunate and the C/C++ function you are using returns an error code, then make use of it. For example:

external_function32 Message_Beep "MessageBeep" user32.dll ;
    word iSound returns integer

integer iErrorCode
move (Message_Beep(-1)) to iErrorCode
if iErrorCode ne 0 begin
    error 201 "Message Beep had an error."
end

The second possibility is obviously a logic error, since you would have received a DataFlex error, otherwise.

Since the first and second cases of error are so hard to determine, your wisest course will be to check if the DLL is being called
properly. In fact, this has been the major focus of this paper. Here is a checklist summarizing the topics covered

  • Make sure the DLL name and DLL function are correct in your DataFlex import statement. Remember that function
    names can be 'mangled' and that two DLLs of the same name can be on your machine (see "Mangled Names in C/C++",
    above). Remember that the C/C++ import name is case-sensitive (and always upper case in 16-bit DLLs.) Also,
    the actual function name may not be the documented one (see "Actual Function Names vs. Documented", above).
  • Make sure the parameter list in your DataFlex import statement is correct. Remember that C/C++ has different sizes
    for integers.
  • If strings or structures are involved, make sure the string has been initialized and the pointer has been obtained before each function call. Failing to do so has been one of the largest sources of errors for our developers. Remember that
    DataFlex may reallocate the memory of a local string every time its value changes.
  • When defining structures, make sure the size of each field is correct. Remember that some C/C++ types have different
    sizes depending on the memory model.    Be sure the import method, external_function,
    external_function16, or external_function32, matches the memory model the DLL uses.

Once you are certain the DLL function is being called correctly, you can concentrate at the C/C++ level and on checking the logic of your DataFlex code.

This document is property of Data Access Corporation. With credit to Data Access Corporation for its authorship, you are
encouraged to reproduce this information in any format either on paper or electronically, in whole or in part. You may publish
this paper as a stand alone document within your own publications conditional on the maintenance of the intent, context, and integrity of the material as it is presented here.