Package: src/packages/dynlink.fdoc
Dynamic Linker¶
key | file |
---|---|
flx_dl.h | share/lib/rtl/flx_dl.h |
flx_dlopen.hpp | share/lib/rtl/flx_dlopen.hpp |
flx_dlopen.cpp | share/src/dynlink/flx_dlopen.cpp |
flx_dynlink.hpp | share/lib/rtl/flx_dynlink.hpp |
flx_dynlink.cpp | share/src/dynlink/flx_dynlink.cpp |
dynlink.flx | share/lib/std/program/dynlink.flx |
config_unix_dl.fpc | $PWD/src/config/unix/dl.fpc |
config_macosx_dl.fpc | $PWD/src/config/macosx/dl.fpc |
config_win_dl.fpc | $PWD/src/config/win/dl.fpc |
unix_flx_dynlink.fpc | $PWD/src/config/unix/flx_dynlink.fpc |
win_flx_dynlink.fpc | $PWD/src/config/win/flx_dynlink.fpc |
flx_dynlink_config.hpp | share/lib/rtl/flx_dynlink_config.hpp |
flx_dynlink.py | $PWD/buildsystem/flx_dynlink.py |
Dynamic Linkage¶
Synopsis¶
This subsystem provides the ability to load,link or otherwise
access program code at run time. We use the name DLL
to refer
to a dynamically loaded file containing executable instructions,
on Windows this is a dynamic link library which usually ends
in extension .dll
whilst on Linux we have shared libraries
with extension .so
and on OSX we use files with extension .dylib
.
There is a confusing array of operations provided here which will require refactoring in the future.
The core platform dependent operations are implemented in C++ and configuration and compile time choices determine the platform supported.
These core operations are wrapped, in C++, to remove the file loading dependencies, and provide resource control integrated with the garbage collector.
We use LoadLibrary
on Windows and dlopen
on Unix platforms
wrapped inside a C++ class flx_dynlink_t
that represents
a library, in Felix the type flx_library
is used.
Felix generated code does not permit variables to be stored in static storage. Instead, a structure is used to contain Felix top level variables. For historical reasons objects are called <em>thread frames.</em>
Members of a thread frame are accessed in Felix bindings
to C++ using the macro PTF
which stands for <em>pointer
to thread frame.</em>
A Felix generated shared library requires an instance to be
created which is a closure: a pair consisting of the library code
and a thread_frame_t
object which is allocated on the heap.
The closure object has the type flx_libinst_t
in C++
and flx_instance
in Felix.
Instances require a fixed protocol which involves
the library containing exported symbols which can
be linked using LoadProcAddress
on Windows
or dlsym
on unix, which can be used to construct
the required thread frame. High level Felix functions
require he thread frame because it contains a pointer
to the garbage collector which in turn provides the
system allocator.
Higher level abstractions require more fixed symbols. In particular, there is a protocol for loading a special kind of library called a <em>plugin</em> which make separate compilation of and use of dynamic libraries particularly convenient.
Dynamic link portability layer¶
A low level layer proving OS and some link technology portability.
The flx_dl.h header: portability macros.¶
//[flx_dl.h]
#ifndef __FLX_DL_H__
#define __FLX_DL_H__
// define dynamic library loader stuff, even for static linkage
// SPECS:
//
// FLX_LIBHANDLE is the type of a native DLL handle
// it's a typedef NOT a macro.
//
// FLX_NOLIBRARY is a macro specifying the value if there is no library.
// used for initialisation, or, error value if dynamic load fails
//
// FLX_LIB_EXTENSION is a macro specifying the string name of
// the platform library extension including the dot (.)
//
// FLX_ENV_LIBRARY_PATH_NAME is a macro that specifies the name
// of the environment variable specifying extra directories
// to search for DLLs.
//
// FLX_NATIVE_DLSYM(lib,sym) accepts a library handle and an identifier.
// It works for both static and dynamic linkage.
//
// For dynamic linkage it converts the symbol to a string
// and calls dlsym
// For static linkage it just returns the provided symbol,
// which should have been linked by the linker.
// This will work for both static linkage AND for
// load time dynamic linkage (but not run time linkage).
//
// FLX_NATIVE_SDLSYM(lib,string) accepts a library handle
// and a string. If lib is a valid run time loaded library,
// this routine works independently of how it was linked
// (since even statically linked programs can dlopen libraries).
//
// It may even work for lib=NULL if the linker is set to export
// symbols to the program, and NULL is the linkers code for the
// module's namespace.
//
// FLX_DLSYM(lib,sym) is just FLX_NATIVE_DLSYM, it requires a symbol.
//
// FLX_SDLSYM(lib,string) uses FLX_NATIVE_SDLSYM if dynamic linkage is selected
// and throws an exception if static linkage is chosen.
//
// Therefore:
// * the "S" version of these macros uses a string name,
// the non-"S" version uses an identifier.
//
// * FLX_NATIVE_SDLSYM uses a string name and always does
// run time lookup.
//
// * FLX_DLSYM uses a symbol and uses a linker bound
// address if FLX_STATIC_LINK is selected
// Otherwise it uses run time lookup.
//
#if FLX_WIN32
#include <windows.h>
typedef HMODULE FLX_LIBHANDLE;
#define FLX_LIB_EXTENSION ".DLL"
#define FLX_NATIVE_DLSYM(x,y) (void*)GetProcAddress(x,#y)
#define FLX_NATIVE_SDLSYM(x,y) (void*)GetProcAddress(x,y)
#define FLX_ENV_LIBRARY_PATH_NAME "PATH"
#else
// UNIX, recent OSX
typedef void *FLX_LIBHANDLE;
#if FLX_CYGWIN
#define FLX_LIB_EXTENSION ".dll"
#define FLX_ENV_LIBRARY_PATH_NAME "LD_LIBRARY_PATH"
#elif FLX_MACOSX
#define FLX_LIB_EXTENSION ".dylib"
#define FLX_ENV_LIBRARY_PATH_NAME "DYLD_LIBRARY_PATH"
#else
#define FLX_LIB_EXTENSION ".so"
#define FLX_ENV_LIBRARY_PATH_NAME "LD_LIBRARY_PATH"
#endif
#include <dlfcn.h>
#define FLX_NATIVE_DLSYM(x,y) dlsym(x,#y)
#define FLX_NATIVE_SDLSYM(x,y) dlsym(x,y)
#endif
#define FLX_NOLIBRARY NULL
#define FLX_DLSYM(x,y) FLX_NATIVE_DLSYM(x,y)
#ifndef FLX_STATIC_LINK
#define FLX_SDLSYM(x,y) FLX_NATIVE_SDLSYM(x,(y))
#else
#define FLX_SDLSYM(x,y) (throw ::flx::rtl::flx_link_failure_t(\
"<static link>",y,"dlsym with static link requires name not string"),\
(void*)0\
)
#endif
#endif
The flx_dlopen
unit: C++ header file.¶
This file contains portable versions of the low level dlopen/LoadLibrary functions.
//[flx_dlopen.hpp]
#ifndef __FLX_DLOPEN_H__
#define __FLX_DLOPEN_H__
#include "flx_dynlink_config.hpp"
#include "flx_dl.h"
#include <string>
using namespace std;
namespace flx { namespace dynlink {
/// Load library
DYNLINK_EXTERN FLX_LIBHANDLE flx_load_library_nothrow(const ::std::string& filename);
DYNLINK_EXTERN FLX_LIBHANDLE flx_load_library_throw(const ::std::string& filename);
DYNLINK_EXTERN FLX_LIBHANDLE flx_load_module_nothrow(const ::std::string& filename);
DYNLINK_EXTERN FLX_LIBHANDLE flx_load_module_throw(const ::std::string& filename);
DYNLINK_EXTERN ::std::string flx_lib_extension ();
DYNLINK_EXTERN ::std::string flx_env_library_path_name ();
DYNLINK_EXTERN FLX_LIBHANDLE flx_nolibrary();
DYNLINK_EXTERN void *flx_native_dlsym
(FLX_LIBHANDLE,::std::string);
}}
#endif
The flx_dlopen
unit: C++ implementation.¶
Implement the RTL portable low level dlopen/LoadLibrary functions.
//[flx_dlopen.cpp]
#include "flx_dlopen.hpp"
#include "flx_exceptions.hpp"
#include <cstdlib>
#include <stdio.h>
namespace flx { namespace dynlink {
FLX_LIBHANDLE
flx_load_library_nothrow(const std::string& filename)
{
FLX_LIBHANDLE library = FLX_NOLIBRARY;
if (::std::getenv("FLX_SHELL_ECHO")!=(char*)0)
fprintf(stderr,"[load_library] %s\n", filename.c_str());
#if FLX_WIN32
// stop windows showing err dialogues, ignoring error code.
(void)SetErrorMode(SEM_NOOPENFILEERRORBOX);
library = LoadLibrary(filename.c_str());
#else
library = dlopen(filename.c_str(),RTLD_NOW | RTLD_LOCAL);
#endif
return library;
}
FLX_LIBHANDLE
flx_load_library_throw(const ::std::string& filename)
{
FLX_LIBHANDLE library = flx_load_library_nothrow(filename);
if(library == FLX_NOLIBRARY)
throw ::flx::rtl::flx_link_failure_t(filename,"LoadLibrary/dlopen","Cannot find dll/shared library");
return library;
}
FLX_LIBHANDLE
flx_load_module_nothrow(const ::std::string& filename)
{
return flx_load_library_nothrow(filename + FLX_LIB_EXTENSION);
}
FLX_LIBHANDLE
flx_load_module_throw(const ::std::string& filename)
{
return flx_load_library_throw(filename + FLX_LIB_EXTENSION);
}
::std::string flx_lib_extension () { return FLX_LIB_EXTENSION; }
::std::string flx_env_library_path_name () { return FLX_ENV_LIBRARY_PATH_NAME; }
FLX_LIBHANDLE flx_nolibrary() { return FLX_NOLIBRARY; }
void *flx_native_dlsym(FLX_LIBHANDLE lib, ::std::string symname)
{
return FLX_NATIVE_DLSYM(lib,symname.c_str());
}
}} // namespaces
Felix level dynamic loader system¶
This is a higher level loader which is primarily designed for loading Felix programs machined as libraries, but it can also be used for high level libraries such as plugins.
The core concept is based on Windows 3.1, in which the library is read only program code, and requires an data frame to execute. Unlike C style libraries, mutable data is not permitted in libraries. Instead, the library must provide a function to create a heap allocated data frame to store global data.
Hence, after loading, one or more instances of the library can be created by combining the code API with a data frame. Felix calls this data frame the <em>thread frame</em>.
Since each client of a library create their own instance of the library, the global variables of the client do not interfere.
The type flx_dynlink_t
represents a library, whereas
the type flx_libinst_t
represents a pair consisting
of the library together with a data frame. This provides
a single entity from which to dispatch function calls
which may interact by per instance data without clobbering
an independent client’s use of the library.
Except in special circumstances Felix demands all code be reentrant and in particular mutable global variables are not allowed at the C level.
The special circumstances are dictated by poor quality
API’s including Posix signals and of course the
notorious errno
.
//[flx_dynlink.hpp]
#ifndef __FLX_DYNLINK_H__
#define __FLX_DYNLINK_H__
#include "flx_rtl.hpp"
#include "flx_gc.hpp"
#include "flx_dl.h"
#include "flx_dlopen.hpp"
#include "flx_exceptions.hpp"
#include "flx_continuation.hpp"
#include <string>
namespace flx { namespace dynlink {
struct DYNLINK_EXTERN flx_dynlink_t;
struct DYNLINK_EXTERN flx_libinst_t;
/// frame creators.
typedef void *(*thread_frame_creator_t)
(
::flx::gc::generic::gc_profile_t*,
void*
);
/// library initialisation routine.
typedef ::flx::rtl::con_t *(*start_t)
(
void*,
int,
char **,
FILE*,
FILE*,
FILE*
);
typedef ::flx::rtl::con_t *(*main_t)(void*);
/// dynamic object loader.
struct DYNLINK_EXTERN flx_dynlink_t
{
// filename of library used for dynamic linkage
::std::string filename;
// modulename of library
// usually filename without path prefix or extension
::std::string modulename;
// OS specific handle refering to the library if one is loaded
// undefine otherwise
FLX_LIBHANDLE library;
// Felix specific entry point used to create thread frame.
// Typically this function allocates the thread frame as a C++
// object, calling its contructor.
// A library together with a thread frame is known as an instance
// of the library.
thread_frame_creator_t thread_frame_creator;
// Felix specific entry point used to initialise thread frame
// Morally equivalent to the body of a C++ constructor,
// this calls the libraries initialisation routine.
// If the library is meant to be a program, this routine
// often contains the program code.
start_t start_sym;
// A separate mainline, morally equivalent to C main() function.
// Intended to be called after the start routine has completed.
main_t main_sym;
// Allow a default initialised default object refering to no library.
flx_dynlink_t(bool debug);
// set static link data into an empty dynlink object.
void static_link(
::std::string modulename,
thread_frame_creator_t thread_frame_creator,
start_t start_sym,
main_t main_sym);
// initialise for static link
// equivalent to default object followed by call to static_link method
flx_dynlink_t(
::std::string modulename,
thread_frame_creator_t thread_frame_creator,
start_t start_sym,
main_t main_sym,
bool debug
) throw(::flx::rtl::flx_link_failure_t);
// dynamic link library from filename and module name
void dynamic_link_with_modulename(
const ::std::string& filename,
const ::std::string& modulename) throw(::flx::rtl::flx_link_failure_t);
// With this variant the module name is calculated from the filename.
void dynamic_link(const ::std::string& filename) throw(::flx::rtl::flx_link_failure_t);
virtual ~flx_dynlink_t();
bool debug;
private:
void unlink(); // implementation of destructor only
flx_dynlink_t(flx_dynlink_t const&); // uncopyable
void operator=(flx_dynlink_t const&); // uncopyable
};
/// Thread Frame Initialisation.
struct DYNLINK_EXTERN flx_libinst_t
{
void *thread_frame;
::flx::rtl::con_t *start_proc;
::flx::rtl::con_t *main_proc;
flx_dynlink_t *lib;
::flx::gc::generic::gc_profile_t *gcp;
void *world; // FIXME: flx_world*, can't specify atm due to circularity
bool debug;
void create
(
flx_dynlink_t *lib_a,
::flx::gc::generic::gc_profile_t *gcp_a,
void *world_a, // FIXME as above
int argc,
char **argv,
FILE *stdin_,
FILE *stdout_,
FILE *stderr_,
bool debug_
);
void destroy ();
::flx::rtl::con_t *bind_proc(void *fn, void *data);
virtual ~flx_libinst_t();
flx_libinst_t(bool debug);
private:
flx_libinst_t(flx_libinst_t const&);
void operator=(flx_libinst_t const&);
};
DYNLINK_EXTERN extern ::flx::gc::generic::gc_shape_t flx_dynlink_ptr_map;
DYNLINK_EXTERN extern ::flx::gc::generic::gc_shape_t flx_libinst_ptr_map;
}} // namespaces
#endif
The flx_dynlink
unit: flx_dynlink_t
class implementation.¶
//[flx_dynlink.cpp]
#include "flx_dynlink.hpp"
#include "flx_strutil.hpp"
#include <stdio.h>
#include <cstring>
#include <cstdlib>
#include <stddef.h>
namespace flx { namespace dynlink {
flx_dynlink_t::flx_dynlink_t(flx_dynlink_t const&) {} // no copy hack
void flx_dynlink_t::operator=(flx_dynlink_t const&) {} // no copy hack
flx_dynlink_t::flx_dynlink_t(bool debug_):
filename(""),
modulename(""),
library(0),
thread_frame_creator(NULL),
start_sym(NULL),
main_sym(NULL),
debug(debug_)
{}
flx_dynlink_t::flx_dynlink_t(
::std::string modulename_a,
thread_frame_creator_t thread_frame_creator,
start_t start_sym,
main_t main_sym,
bool debug_
) throw(::flx::rtl::flx_link_failure_t)
:
modulename (modulename_a),
library(0),
thread_frame_creator(thread_frame_creator),
start_sym(start_sym),
main_sym(main_sym),
debug(debug_)
{
if(!thread_frame_creator)
throw ::flx::rtl::flx_link_failure_t("<static link>","dlsym","create_thread_frame");
if(!start_sym)
throw ::flx::rtl::flx_link_failure_t("<static link>","dlsym","flx_start");
}
void flx_dynlink_t::static_link (
::std::string modulename,
thread_frame_creator_t thread_frame_creator,
start_t start_sym,
main_t main_sym
)
{
this->modulename = modulename;
this->thread_frame_creator = thread_frame_creator;
this->start_sym = start_sym;
this->main_sym = main_sym;
}
void flx_dynlink_t::dynamic_link_with_modulename(const ::std::string& filename_a, const ::std::string& modulename_a) throw(::flx::rtl::flx_link_failure_t)
{
filename = filename_a;
modulename = modulename_a;
library = flx_load_library_throw(filename);
//fprintf(stderr,"File %s dlopened at %p ok\n",fname.c_str(),library);
thread_frame_creator = (thread_frame_creator_t)
FLX_NATIVE_SDLSYM(library,(modulename+"_create_thread_frame").c_str());
if(!thread_frame_creator)
throw ::flx::rtl::flx_link_failure_t(filename,"dlsym",modulename+"_create_thread_frame");
if (debug)
fprintf(stderr,"[dynlink:dynamic_link] Thread frame creator found at %p\n",thread_frame_creator);
start_sym = (start_t)FLX_NATIVE_SDLSYM(library,(modulename+"_flx_start").c_str());
if (debug)
fprintf(stderr,"[dynlink:dynamic_link] Start symbol = %p\n",start_sym);
if(!start_sym)
throw ::flx::rtl::flx_link_failure_t(filename,"dlsym",modulename+"_flx_start");
main_sym = (main_t)FLX_NATIVE_SDLSYM(library,"flx_main");
if(debug)
fprintf(stderr,"[dynlink:dynamic_link] main symbol = %p\n",main_sym);
}
void flx_dynlink_t::dynamic_link(const ::std::string& filename_a) throw(::flx::rtl::flx_link_failure_t)
{
string mname = ::flx::rtl::strutil::filename_to_modulename (filename_a);
dynamic_link_with_modulename(filename_a,mname);
}
// dont actually unload libraries
// it doesn't work right in C/C++
// can leave dangling references
// impossible to manage properly
void flx_dynlink_t::unlink()
{
//fprintf(stderr,"closing library\n");
//#if FLX_WIN32 || FLX_CYGWIN
#if FLX_WIN32
//FreeLibrary(library);
#else
//dlclose(library);
#endif
}
flx_dynlink_t::~flx_dynlink_t() {
// fprintf(stderr, "Library %p of module '%s' file '%s' destroyed\n", this,
// modulename.c_str(), filename.c_str()
// );
}
The flx_dynlink
unit: flx_libinst_t
class implementation.¶
//[flx_dynlink.cpp]
// ************************************************
// libinst
// ************************************************
flx_libinst_t::~flx_libinst_t() {
// fprintf(stderr, "Library instance %p of library %p destroyed\n",this,lib);
}
flx_libinst_t::flx_libinst_t(bool debug_) :
thread_frame (NULL),
start_proc (NULL),
main_proc (NULL),
lib (NULL),
gcp(NULL),
debug(debug_)
{}
flx_libinst_t::flx_libinst_t(flx_libinst_t const&){}
void flx_libinst_t::operator=(flx_libinst_t const&){}
void flx_libinst_t::create
(
flx_dynlink_t *lib_a,
flx::gc::generic::gc_profile_t *gcp_a,
void *world_a,
int argc,
char **argv,
FILE *stdin_,
FILE *stdout_,
FILE *stderr_,
bool debug_
)
{
lib = lib_a;
gcp = gcp_a;
world = world_a;
debug = debug_;
if (debug)
fprintf(stderr,"[libinst:create] Creating instance for library %p->'%s'\n",lib, lib->filename.c_str());
if (debug)
fprintf(stderr, "[libinst:create] Creating thread frame\n");
thread_frame = lib->thread_frame_creator( gcp, world);
if (debug)
fprintf(stderr, "[libinst:create] thread frame CREATED %p\n", thread_frame);
if (debug)
fprintf(stderr, "[libinst:create] CREATING start_proc by running start_sym %p\n", lib->start_sym);
try {
start_proc = lib->start_sym(thread_frame, argc, argv, stdin_,stdout_,stderr_);
}
catch (::flx::rtl::con_t *p) {
if (debug)
fprintf(stderr,
"[lininst::create] setting start_proc to continuation %p thrown by start_sym %p\n",
p,lib->start_sym);
start_proc = p;
}
if (debug)
fprintf(stderr, "[libinst:create] start_proc CREATED %p\n", start_proc);
if (debug)
fprintf(stderr, "[libinst:create] CREATING main_proc by running main_sym %p\n", lib->main_sym);
main_proc = lib->main_sym?lib->main_sym(thread_frame):0;
if (debug)
fprintf(stderr, "[libinst:create] main_proc CREATED %p\n", main_proc);
}
::flx::rtl::con_t *flx_libinst_t::bind_proc(void *fn, void *data) {
typedef ::flx::rtl::con_t *(*binder_t)(void *,void*);
return ((binder_t)fn)(thread_frame,data);
}
// ********************************************************
// OFFSETS for flx_dynlink_t
// ********************************************************
FLX_FINALISER(flx_dynlink_t)
::flx::gc::generic::gc_shape_t flx_dynlink_ptr_map = {
"dynlink::flx_dynlink_t",
1,sizeof(flx_dynlink_t),
flx_dynlink_t_finaliser,
0, // fcops
0, // private data
0, // scanner
::flx::gc::generic::tblit<flx_dynlink_t>, // encoder
::flx::gc::generic::tunblit<flx_dynlink_t>, // decoder
::flx::gc::generic::gc_flags_default, // flags
0UL, 0UL
};
// ********************************************************
// OFFSETS for flx_libinst
// ********************************************************
static const ::flx::gc::generic::offset_entry_t flx_libinst_offsets[4]={
{offsetof(flx_libinst_t,thread_frame),nullptr},
{offsetof(flx_libinst_t,start_proc),nullptr},
{offsetof(flx_libinst_t,main_proc),nullptr},
{offsetof(flx_libinst_t,lib),nullptr}
};
FLX_FINALISER(flx_libinst_t)
static ::flx::gc::generic::offset_data_t const flx_libinst_offset_data = { 4, flx_libinst_offsets };
::flx::gc::generic::gc_shape_t flx_libinst_ptr_map = {
"dynlink::flx_libinst",
1,sizeof(flx_libinst_t),
flx_libinst_t_finaliser,
0, // fcops
&flx_libinst_offset_data,
::flx::gc::generic::scan_by_offsets,
::flx::gc::generic::tblit<flx_libinst_t>,::flx::gc::generic::tunblit<flx_libinst_t>,
::flx::gc::generic::gc_flags_default,
0UL, 0UL
};
}} // namespaces
The dynamic link library binding Dynlink
¶
//[dynlink.flx]
class Dynlink
{
C++ support package.¶
//[dynlink.flx]
requires package "flx_dynlink";
Error handling.¶
The current version of the library requires dynamic link attempts to succeed. If they fail an exception is thrown which aborts the program unless specifically caught. In future, we may provide an interface based on option types which enforces user level error checking as well.
//[dynlink.flx]
//$ Exception thrown if dynamic linkage fails.
type flx_link_failure_t = "::flx::rtl::flx_link_failure_t";
//$ Constructor for dynamic linkage exception.
ctor flx_link_failure_t : string * string * string = "::flx::rtl::flx_link_failure_t($1,$2,$3)";
//$ Extractors.
fun filename : flx_link_failure_t -> string = "$1.filename";
fun operation : flx_link_failure_t -> string = "$1.operation";
fun what : flx_link_failure_t -> string = "$1.what";
//$ Delete returned exception.
proc delete : cptr[flx_link_failure_t] = "delete $1;";
//$ This doesn't belong here but it will do for now
fun get_debug_driver_flag : 1 -> bool = "PTF gcp->debug_driver" requires property "needs_gc";
Library handle flx_library
¶
A platform independent handle which can refer to a dynamic link library object. Operations in this category are universal and apply to all dynamic link libraries, whether or not they were generated by Felix.
//[dynlink.flx]
//$ Type of a DLL (dynamic link library) object.
_gc_pointer type flx_library = "::flx::dynlink::flx_dynlink_t*";
Constructor for flx_library</code>: <code>create_library_handle
¶
The constructor makes an unpopulated library handle not associated with any particular DLL.
//[dynlink.flx]
//$ Create a fresh DLL object.
fun create_library_handle: bool ->flx_library=
"new(*PTF gcp, ::flx::dynlink::flx_dynlink_ptr_map, false) ::flx::dynlink::flx_dynlink_t($1)";
Load a library dlopen
¶
This procedure associates a library handle with a particular file name and also attempts to load the library.
//[dynlink.flx]
//$ Link a DLL using given filename.
//$ May throw flx_link_failure_t.
proc dlopen:flx_library * string = "$1->dynamic_link($2);";
//$ Link a DLL using given filename and modulename.
//$ May throw flx_link_failure_t.
proc modopen:flx_library * string * string =
"$1->dynamic_link_with_modulename($2, $3);"
;
//$ Link static
proc set_entry_points : flx_library * string * address * address =
"$1->static_link($2,(::flx::dynlink::thread_frame_creator_t)$3, (::flx::dynlink::start_t)$4, NULL);"
;
Load a library from registry regopen
¶
Given a registry, simulate dynamic linkage.
//[dynlink.flx]
typedef module_dictionary_t = StrDict::strdict[address];
typedef registry_t = StrDict::strdict[module_dictionary_t];
fun get_module_registry_address_address: 1 -> &®istry_t =
"(void****)(void*)&(PTF gcp->collector->module_registry)"
requires property "needs_gc";
// severe hackery: if the registry isn't initialised,
// create one, store its address in the GC object, and make
// it a root so the GC scans it: the GC isn't owned by itself,
// but the registry is owned by the GC.
gen get_module_registry () :registry_t = {
var ppregistry : &®istry_t = #get_module_registry_address_address;
var pregistry : ®istry_t = *ppregistry;
if C_hack::isNULL (pregistry) do
pregistry = new (StrDict::strdict[module_dictionary_t] ());
ppregistry <- pregistry;
Gc::add_root (C_hack::cast[address] (pregistry));
done
return *pregistry;
}
noinline proc regopen (registry:registry_t) (lib:flx_library, modulename:string)
{
//println$ "regopen " + modulename;
var mod = StrDict::get registry modulename;
match mod with
| #None =>
//println$ "Not in registry, using dlopen for " + modulename;
modopen$ lib, modulename+#Filename::dynamic_library_extension, modulename;
| Some dict =>
//println$ "Found module "+modulename+" in registry";
var tfc = dict.get_dflt (modulename+"_create_thread_frame", NULL);
//println$ "Thread frame creator = " + str tfc;
if tfc == NULL do
raise$ flx_link_failure_t(modulename,"regopen","Cannot find symbol " + modulename+"_create_thread_frame in module registry for " + modulename);
done
var start_sym = dict.get_dflt (modulename+"_flx_start",NULL);
if start_sym == NULL do
raise$ flx_link_failure_t(modulename,"regopen","Cannot find symbol " + modulename+"_flx_start in module registry for "+modulename);
done
//println$ "Start symbol = " + str start_sym;
set_entry_points$ lib,modulename,tfc, start_sym;
endmatch;
}
Get the filename associated with a library handle: filename
¶
//[dynlink.flx]
//$ Get the filename of a DLL.
fun filename : flx_library -> string = "$1->filename";
//$ Get the modulename of a DLL.
fun modulename : flx_library -> string = "$1->modulename";
//$ Get the threadframe creator function
fun get_thread_frame_creator_as_address: flx_library -> address = "(void*)$1->thread_frame_creator";
//$ Get start function
fun get_start_as_address: flx_library -> address = "(void*)$1->start_sym";
noinline proc add_symbol (modulename:string, symbolname:string, adr:address)
{
//println$ "add symbol " + symbolname + " to module " + modulename+ " value " + str adr;
var registry = #Dynlink::get_module_registry;
var mod = #{
match get registry modulename with
| #None =>
var mod = #strdict[address];
add registry modulename mod;
return mod;
| Some dict => return dict;
endmatch;
};
mod.add symbolname adr;
}
Unlink a dll : dlclose
.¶
This routine reduces the reference count of a library handle by one, and if it drops to zero unloads the library at the OS level.
References counts are increase by one when instances are created.
The initial dlopen
sets the reference count to 1.
Unlinking clears the association of the handle with the filename and tells the platform linker to unlink the library. However this does not necessarily unload the library because the platform linker may also reference count the library, and the user may link the same DLL twice using distinct library handles.
Because of the badly designed structure of C programs, unloading a library physically is not safe and cannot be made safe. Even with tight control of library code generation, it is very hard to ensure there are no references left to a library. References include pointers to functions, vtables, rtti objects, strings, other constants, and sometimes even variables.
//[dynlink.flx]
//$ Unlink a DLL.
//$ Unsafe! Use with extreme caution.
//$ May cause pointers into the DLL code segment to dangle.
proc dlclose:flx_library = "$1->unlink();";
Get the address of an exported symbol: dlsym
¶
This routine takes a library and a string argument
and tries to find the value associated with the string
in the library symbol table, using GetProcAddress
on Windows or dlsym
on Unix. This action is independent
of whether the calling program was linked dynamically
or statically.
For functions, this operator returns a function pointer. For variables, it returns the address of the variable. DO not forget the extra dereference requires if the variable is itself a pointer.
//[dynlink.flx]
//$ Find raw address of a symbol in a DLL.
//$ This function now ALWAYS does a dlsym
//$ (or Windows equivalent)
//$ even for static linkage: after all
//$ statically linked executables can still
//$ load DLLs at run time.
fun raw_dlsym:flx_library * string->address =
"FLX_NATIVE_SDLSYM($1->library,$2.c_str())";
noinline fun find_sym(lib:flx_library, sym:string) : address =
{
if lib.filename == "" do
var reg = #get_module_registry;
match reg.get lib.modulename with
| #None => return NULL;
| Some dict =>
match dict.get sym with
| #None => return NULL;
| Some sym => return sym;
endmatch;
endmatch;
else
return raw_dlsym (lib,sym);
done
}
Library instance type flx_instance
¶
A library instance is a closure consisting of the library code, represent by a library handle, together with a pointer to an instance of the libraries thread frame. Operations in this category only work with Felix generated library objects.
//[dynlink.flx]
//$ Type of a DLL (dynamic link library) instance.
//$ Conceptually this is a pair consisting of
//$ a library object and a global data frame object.
_gc_pointer type flx_instance = "::flx::dynlink::flx_libinst_t*";
Library instance constructor create_instance_handle
¶
Create a new library instance handle unassociated with any library or thread frame.
//[dynlink.flx]
//$ Create a fresh DLL instance object.
fun create_instance_handle: bool->flx_instance=
"new(*PTF gcp, ::flx::dynlink::flx_libinst_ptr_map, false) ::flx::dynlink::flx_libinst_t($1)";
Create a library instance from a library: create
¶
This procedure creates a thread frame from a library, initialises it, and sets the given library instance with the library handle and thread frame. The instance handle should not already be associated with a library or thread frame.
//[dynlink.flx ]
//$ Create a DLL instance from a DLL.
//$ This is a procedure, so maybe the caller is too
//$ which means the thread frame must be available.
proc create: flx_library * flx_instance =
"$2->create($1,PTF gcp,PTF world,PTF argc,PTF argv,PTF flx_stdin, PTF flx_stdout, PTF flx_stderr, false);"
requires property "needs_gc"
;
proc create_with_args: flx_library * flx_instance * int * + (+char) =
"$2->create($1,PTF gcp,PTF world,$3,$4,PTF flx_stdin, PTF flx_stdout, PTF flx_stderr, false);"
requires property "needs_gc"
;
proc create_with_args (lib:flx_library, inst:flx_instance, args:list[string])
{
// convert list to a varray of strings
var a = varray args;
// now convert to varray of char pointers
gen myget(i:size)=>a.i.cstr;
var x = varray[+char] (a.len,a.len,myget);
create_with_args (lib,inst,x.len.int,x.stl_begin);
}
Get the filename from an instance filename
.¶
//[dynlink.flx ]
//$ Get the filename of a DLL from an instance of it.
fun filename : flx_instance -> string = "::std::string($1->lib->filename)";
Get the startup procedure from an instance filename
.¶
Felix generated libraries contain a symbol which is used to initialise the thread frame. This initialisation is in addition to that performed when the instance is created. Typically the instance creation initialisation simply invokes the C++ default constructor and sets a couple of critical variables including a pointer to the garbage collector and standard input/output streams.
The startup procedure we get here, on the other hand, usually refers to the client program when using the scripting model; the behaviour of the program is the observable side effects of this initialisation procedure.
For plugin libraries, the initialisation procedure is used to construct default values or initialise starting state.
The initialisation procedure is represent by a pointer
to a continuation object, type cont
, which has to be run
by a scheduler after associating it with a fibre:
the procedure is <em>not</em> a C function.
//[dynlink.flx ]
//$ Get the initialisation continuation of an instance.
fun get_init: flx_instance -> cont = "$1->start_proc";
Get the library associated with an instance.¶
//[dynlink.flx]
//$ Get the DLL associated with an instance.
fun get_library: flx_instance -> flx_library = "$1->lib";
Get the thread frame associated with an instance.¶
Since we don’t know the type of the thread frame here, it is returned as a pure address.
//[dynlink.flx]
//$ Get the thread frame (global data object) of an instance.
fun get_thread_frame: flx_instance -> address = "(void*)$1->thread_frame";
Convenience constructor for an instance init_lib
¶
This function creates a library handle and instance handle and loads the library given a filename, all in one operation. Then it runs the startup initialisation procedure. Finally the instance is returned.
//[dynlink.flx]
//$ Create, link, and prepare a DLL instance from a modulename.
//$ NOTE: libraries created here do not need to be roots
// The code is never deleted (due to design issues with C).
// If the library isn't reachable, you can't create an instance.
// If an instance is created, it reaches the library.
noinline gen prepare_lib(modulename:string):flx_instance = {
var dlibrary = create_library_handle(get_debug_driver_flag());
//Gc::add_root (C_hack::cast[address] library);
var linstance = create_instance_handle(get_debug_driver_flag());
regopen #get_module_registry (dlibrary,modulename);
create (dlibrary,linstance);
return linstance;
}
//$ Create, link, and prepare a DLL instance from a modulename.
noinline gen prepare_lib_with_args(modulename:string, args:list[string]):flx_instance = {
var dlibrary = create_library_handle(get_debug_driver_flag());
//Gc::add_root (C_hack::cast[address] library);
var linstance = create_instance_handle(get_debug_driver_flag());
regopen #get_module_registry (dlibrary,modulename);
create_with_args (dlibrary,linstance,args);
return linstance;
}
//$ Create, link, and initialise a DLL instance from a modulename.
noinline gen init_lib(modulename:string):flx_instance = {
var linstance = prepare_lib(modulename);
var init = get_init linstance;
Fibres::run init;
return linstance;
}
//$ Create, link, and initialise a DLL instance from a modulename.
noinline gen init_lib_with_args(modulename:string, args:list[string]):flx_instance = {
var linstance = prepare_lib_with_args(modulename,args);
var init = get_init linstance;
Fibres::run init;
return linstance;
}
Convenience to run a program run_lib
¶
This function does the same as init_lib
.
//[dynlink.flx]
//$ Run a Felix program from a filename.
proc run_lib(modulename:string)
{
var linstance = init_lib(modulename);
C_hack::ignore(linstance);
}
// BUG: no return code!
proc run_program(args:list[string])
{
match args with
| Cons (h, t) =>
var linstance = prepare_lib_with_args(h,t);
var init = get_init linstance;
Fibres::run init;
| _ => ;
endmatch;
}
Checked version of dlsym
¶
This routine tries to find a symbol with the specified name in an instance, if it is found, the resulting address is cast to the specified type. It also prints a diagnostic if the symbol cannot be found.
//[dynlink.flx]
//$ Find typed address of a symbol in a DLL.
noinline fun flx_dlsym[T] (linst: flx_instance, sym:string) = {
var dlibrary = Dynlink::get_library linst;
var tf = Dynlink::get_thread_frame linst;
//println$ "Trying to load symbol " + sym + " from library " + linst.filename;
var raw_sym = Dynlink::find_sym$ dlibrary, sym;
if isNULL raw_sym do
eprintln$ "Unable to load symbol " + sym + " from library " + linst.filename;
raise$ flx_link_failure_t(linst.filename,"dlsym","Cannot find symbol " + sym);
done
// eprintln$ "loaded symbol " + sym + " from library " + linst.filename + " address= " + str raw_sym;
var typed_sym = C_hack::cast[T] raw_sym;
return typed_sym, tf;
}
Higher level wrappers for finding Felix functions.¶
Here make a set of higher level wrappers for finding standard protocol Felix function in DLLs. These wrappers create a closure by binding the C address of the constructor for the Felix function class in C++ to the library instance, and return that.
Closures returned by these function can be invoked as normal Felix functions and procedures. Whereas a function defined in the current files binds to the thread frame implicitly, with a library the instance is required to supply the thread frame. The closures returned by these wrappers are bound to the libraries thread frame so they can be invoked with the ordinary syntax.
Note that these operations are not type safe. If you get the type wrong all hell will break loose. This is because dlsym finds functions by their C names and C++ entities use mangled names we cannot compute in a portable way.
//[dynlink.flx]
//$ Return a closure representing a symbol in a DLL instance
//$ of a function of no arguments.
noinline fun func0[R] (linst: flx_instance, sym:string) = {
var s,tf= flx_dlsym[address --> R] (linst, sym);
return fun () => s tf;
}
//$ Return a closure representing a symbol in a DLL instance
//$ of a function of one argument.
noinline fun func1[R,A0] (linst: flx_instance, sym:string) = {
var s,tf= flx_dlsym[address * A0 --> R] (linst, sym);
return fun (a0:A0) => s (tf, a0);
}
//$ Return a closure representing a symbol in a DLL instance
//$ of a function of two arguments.
noinline fun func2[R,A0,A1] (linst: flx_instance, sym:string) = {
var s,tf= flx_dlsym[address * A0 * A1 --> R] (linst, sym);
return fun (var a0:A0, var a1:A1) => s (tf, a0, a1);
}
//$ Return a closure representing a symbol in a DLL instance
//$ of a procedure of no arguments.
noinline fun proc0 (linst: flx_instance, sym:string) = {
var s,tf= flx_dlsym[address --> void] (linst, sym);
return proc () { s tf; };
}
//$ Return a closure representing a symbol in a DLL instance
//$ of a procedure of one argument.
noinline fun proc1[A0] (linst: flx_instance, sym:string) = {
var s,tf= flx_dlsym[address * A0 --> void] (linst, sym);
return proc (a0:A0) { s (tf, a0); };
}
//$ Return a closure representing a symbol in a DLL instance
//$ of a procedure of two arguments.
noinline fun proc2[A0,A1] (linst: flx_instance, sym:string) = {
var s,tf= flx_dlsym[address * A0 * A1 --> void] (linst, sym);
return proc (a0:A0,a1:A1) { s (tf, a0, a1); };
}
Plugins.¶
A plugin is a special kind of DLL which supplies two fixed entry points: a setup routine, which is called to initialise the thread frame given a string argument, and a single entry point which is subsequently called and which typically returns an object type consisting of a set of methods acting on the object state and initialised thread frame context.
The setup routine typically take a string of configuration parameters, extracts them with a parser, and stores them in variables.
The current protocol is that the setup function must be called “dllname_setup”, the entry point name is passed as a string.
In order to accomodate static linking of plugins in the future, the setup and entry point symbols would need to have univerally unique names, since static linkage cannot work with duplicate definitions, so the protocol will change to require the library name as a prefix. Stay tuned.
//[dynlink.flx]
//$ Specialised routine(s) to load stylised plugin.
//$ Two entry points:
//$
//$ setup: string -> int
//$
//$ is called to initialise the instance globals.
//$
//$ entry-point: arg -> iftype
//$
//$ is the primary entry point, typically an object factory,
//$ when called with an argument
//$ of type arg_t it returns //$ an object of type iftype.
//$
//$ This function returns the object factory.
//$ setup is called automatically with the supplied string.
//$
//$ There are 3 variants where the factory function accepts
//$ 0, 1 and 2 arguments.
noinline gen load-plugin-func0[iftype] (
dll-name: string, // name of the DLL minus the extension
setup-str: string="", // string to pass to setup
entry-point: string="" // export name of factory function
) : unit -> iftype =
{
var entrypoint = if entry-point == "" then dll-name else entry-point;
var linst = Dynlink::init_lib(dll-name);
var sresult = Dynlink::func1[int,string] (linst, dll-name+"_setup") (setup-str);
C_hack::ignore(sresult);
if sresult != 0 call eprintln$ "[dynlink] Warning: Plugin Library " + dll-name + " set up returned " + str sresult;
return Dynlink::func0[iftype] (linst, entrypoint);
}
noinline gen load-plugin-func1[iftype, arg_t] (
dll-name: string, // name of the DLL minus the extension
setup-str: string="", // string to pass to setup
entry-point: string="" // export name of factory function
) : arg_t -> iftype =
{
var entrypoint = if entry-point == "" then dll-name else entry-point;
var linst = Dynlink::init_lib(dll-name);
var sresult = Dynlink::func1[int,string] (linst, dll-name+"_setup") (setup-str);
C_hack::ignore(sresult);
if sresult != 0 call eprintln$ "[dynlink] Warning: Plugin Library " + dll-name + " set up returned " + str sresult;
return Dynlink::func1[iftype,arg_t] (linst, entrypoint);
}
noinline gen load-plugin-func2[iftype, arg1_t, arg2_t] (
dll-name: string, // name of the DLL minus the extension
setup-str: string="", // string to pass to setup
entry-point: string="" // export name of factory function
) : arg1_t * arg2_t -> iftype =
{
var entrypoint = if entry-point == "" then dll-name else entry-point;
var linst = Dynlink::init_lib(dll-name);
var sresult = Dynlink::func1[int,string] (linst, dll-name+"_setup") (setup-str);
C_hack::ignore(sresult);
if sresult != 0 call eprintln$ "[dynlink] Warning: Plugin Library " + dll-name + " set up returned " + str sresult;
return Dynlink::func2[iftype,arg1_t, arg2_t] (linst, entrypoint);
}
Utilities and misc.¶
//[dynlink.flx]
//$ Execute an address representing a top
//$ level exported felix procedure's C wrapper,
//$ this creates a 'read to run' continuation object
//$ by both constructing the object using the thread
//$ frame of the instance as an argument, and calling
//$ it to fix a null return address and an arbitrary
//$ client data pointer as arguments to the call method.
fun bind_proc: flx_instance * address * address -> cont =
"$1->bind_proc($2,$3)";
//$ Get the OS dependent handle representing a loaded DLL.
//$ Return as an address.
fun dlib_of : flx_library -> address = "(void*)$1->library";
//$ Throw an exception indicating the failure to
//$ find a symbol in a DLL.
proc dlsym_err:flx_library*string="""
throw ::flx::rtl::flx_link_failure_t($1->filename,$2,"symbol not found");
""";
//$ Run a procedure represented by a string name with
//$ given thread frame.
noinline proc run_proc (linstance:flx_instance, p: string, data: address)
{
var lib = get_library linstance;
var sym = find_sym(lib, p);
if isNULL(sym) call dlsym_err(lib,p);
var f = bind_proc(linstance, sym, data);
run f;
}
}
Dynamic Linkage support¶
//[flx_dynlink_config.hpp]
#ifndef __FLX_DYNLINK_CONFIG_H__
#define __FLX_DYNLINK_CONFIG_H__
#include "flx_rtl_config.hpp"
#ifdef BUILD_DYNLINK
#define DYNLINK_EXTERN FLX_EXPORT
#else
#define DYNLINK_EXTERN FLX_IMPORT
#endif
#endif
//[config_unix_dl.fpc]
Name: dl
Description: dynamic loading support
includes: '<dlfcn.h>'
requires_dlibs: -ldl
requires_slibs: -ldl
//[config_macosx_dl.fpc]
Name: dl
Description: dynamic loading support
includes: '<dlfcn.h>'
//[config_win_dl.fpc]
Name: dl
Description: dynamic loading support
//[unix_flx_dynlink.fpc]
Name: flx_dynlink
Description: Felix Dynamic loading support
provides_dlib: -lflx_dynlink_dynamic
provides_slib: -lflx_dynlink_static
Requires: dl flx_exceptions flx_gc flx_strutil
library: flx_dynlink
includes: '"flx_dynlink.hpp"'
macros: BUILD_DYNLINK
srcdir: src/dynlink
src: .*\.cpp
//[win_flx_dynlink.fpc]
Name: flx_dynlink
Description: Felix Dynamic loading support
provides_dlib: /DEFAULTLIB:flx_dynlink_dynamic
provides_slib: /DEFAULTLIB:flx_dynlink_static
Requires: dl flx_exceptions flx_gc flx_strutil
library: flx_dynlink
includes: '"flx_dynlink.hpp"'
macros: BUILD_DYNLINK
srcdir: src/dynlink
src: .*\.cpp
#[flx_dynlink.py]
import fbuild
from fbuild.path import Path
from fbuild.record import Record
from fbuild.builders.file import copy
from fbuild.functools import call
import buildsystem
# ------------------------------------------------------------------------------
def build_runtime(phase):
print('[fbuild] [rtl] build dynlink')
path = Path(phase.ctx.buildroot/'share'/'src/dynlink')
srcs = [f for f in Path.glob(path / '*.cpp')]
includes = [phase.ctx.buildroot / 'host/lib/rtl', phase.ctx.buildroot / 'share/lib/rtl']
macros = ['BUILD_DYNLINK']
libs = [
call('buildsystem.flx_strutil.build_runtime', phase),
call('buildsystem.flx_gc.build_runtime', phase),
]
dst = 'host/lib/rtl/flx_dynlink'
return Record(
static=buildsystem.build_cxx_static_lib(phase, dst, srcs,
includes=includes,
libs=[lib.static for lib in libs],
macros=macros),
shared=buildsystem.build_cxx_shared_lib(phase, dst, srcs,
includes=includes,
libs=[lib.shared for lib in libs],
macros=macros))