Is it possible to call C functions?

[GB18030-2000]

This feature is called Foreign Function Interface (or FFI) in other languages.

[arakov]

Yes, the language has support for calling C functions declared in shared libraries (only C functions so far, maybe later C++ as well).

For example, in system\net\win_sockets.l, a function from WS2_32 shared library is called:

  SOCKADDR_IN addr := SOCKADDR_IN.Default;
  addr.Port := port;
  addr.Family := system'net'AF_INET;
  addr.Ip_address := ip_address;
    
  pointer ptr := addr;
  int retVal := extern WS2_32.connect(_handle, ptr, SOCKADDR_IN_SIZE);

[groggily]

There are still many things not clear. For example, I only need to extern [Dll_name].methods(...) and nothing else? No other boilerplate code?

[groggily]

For example, I want to use the Win32 APIs MultiByteToWideChar() and WideCharToMultiByte(). Can you give me example code?

[arakov]

Yes, it is currently quite simple (but unsafe). The compiler does not check if the function exists or correct signature is used.

Sure, you can take a look at system'text'WinEncoder (src60\system\text\win_encoding.l), it uses exactly these two functions:

     byte buffer[4];
     short tmp[4];
     int retVal := extern KERNEL32.WideCharToMultiByte(
        codePage,  // the coding
        0,
        tmp, // a stack-allocated array of shorts
        tmpLen,  
        buffer,   // a stack-allocated array of bytes
        4, 0, 0);

or

     short tmp[4];
     int byteArrayLen := 1;

     int converted := extern KERNEL32.MultiByteToWideChar(
         codePage, 
         MB_ERR_INVALID_CHARS,  // the constant = 8
         byteArray,  // the byte array, for example byte[] or stack allocated byte byteArray[N]
         byteArrayLen,  
         tmp, 4);

You can skip assigning the result (if for example the function does not return a value)

There is some nuances about the type of arguments. You can ask me how to call other function if you need it as well.

Probably for simplicity, you can give your code snippet, and I can correct if something is wrong

[groggily]

How to handle callback functions? Some C functions expect a function pointer as parameter.

[arakov]

It is a bit tricky. So far I wrote them directly in byte-codes. Now it will be good opportunity to make it available for everyone, please take a look at - #742.

Can you please provide me a signature for a callback function you need? So I can make sure that your code will be supported.

But you have to understand that passing the objects directly is not possible. There are alternatives how to do it. You can allocate an object in PERM memory (as it is done for GUI application) or you can declare a static collection and pass the index to the function, so in your callback you can use this index to resolve the proper object.

[groggily]

I'm translating this C code to ELENA:

#include <stdlib.h>
#include <iup.h>

int btn_count_cb( Ihandle *self )
{
  Ihandle* text = IupGetDialogChild(self, "TEXT");
  int value = IupGetInt(text, "VALUE");
  IupSetInt(text, "VALUE", ++value);

  return IUP_DEFAULT;
}

int main(int argc, char **argv)
{
  Ihandle *dlg, *button, *text, *hbox;

  IupOpen(&argc, &argv);
  
  button = IupButton("Count", NULL);
  IupSetAttribute(button, "SIZE", "60");
  text = IupText(NULL);
  IupSetAttribute(text, "SIZE", "60");
  IupSetAttribute(text, "NAME", "TEXT");
  IupSetAttribute(text, "READONLY", "YES");
  hbox = IupHbox(
    text,
    button,
    NULL);
  IupSetAttribute(hbox, "MARGIN", "10x10");
  IupSetAttribute(hbox, "GAP", "10");
  dlg = IupDialog(hbox);
  IupSetAttribute(dlg, "TITLE", "Counter");

  IupSetInt(text, "VALUE", 0);

  /* Registers callbacks */
  IupSetCallback(button, "ACTION", (Icallback) btn_count_cb);

  IupShowXY(dlg, IUP_CENTER, IUP_CENTER);

  IupMainLoop();

  IupClose();
  return EXIT_SUCCESS;
}

Icallback is defined in iup.h like this:

typedef struct Ihandle_ Ihandle;
typedef int (*Icallback)(Ihandle*);

Download IUP: https://sourceforge.net/projects/iup/files/3.32/Windows%20Libraries/Dynamic/iup-3.32_Win64_dll17_lib.zip/download

[arakov]

I've just implemented the feature. It will be available with a next nightly build. Your callback function seems to be simple one, so you can do similar to my example:

const int CTRL_C_EVENT = 0;
const int CTRL_BREAK_EVENT = 1;

extern int myFunction(uint fdwCtrlType)
{
   fdwCtrlType =>
      CTRL_C_EVENT : { Console.writeLine("Ctrl-C event"); ^ -1 }
      CTRL_BREAK_EVENT : { Console.writeLine("Ctrl-Break event"); ^ 0 };
      
   ^ 0;      
}

public program()
{
   extern KERNEL32.SetConsoleCtrlHandler(&myFunction, -1);
   
   Console.writeLine("The Control Handler is installed.");
   Console.writeLine("-- Now try pressing Ctrl+C or Ctrl+Break");
   
   while(true) {}
}

where myFunction is a callback function

[groggily]

I find out that I have to write extern IUP. before each external function calls and there are too many of them.

Could you introduce a new syntax extension to help with this?

[arakov]

Do you mean providing some short-cut? Something like:

instead of:

 extern DLL.SetMyValue(extern DLL.GetMyValue());

something like

#alias DLL = extern DLL;
// ...    
DLL.SetMyValue(DLL.GetMyValue());

[groggily]

Yes, I mean some kind of a short-cut.

On your sample code above, is #alias an existing syntax of the language or something need to be added?

It's a function alias (very unlikely), type alias (extern DLL is a type represents an external library?), or simply a text substitution?

[groggily]

What about something like this?

extern DLL;
DLL.SetMyValue(DLL.GetMyValue());

If extern DLL; causes ambiguity, what about extern library DLL;?

[arakov]

I think I will implement it like this:

use extern DLL;

I would like to use it for type short-cuts as well:

use system'text'StringBuilder;

so using only a name will be possible

[arakov]

So, I've added a support for that type of shortcut:

use extern KERNEL32;

public program()
{
   pointer commandLinePtr := KERNEL32.GetCommandLineW(); // skipping extern attribute
}

Must be available with a next nightly build. I will do it manually, so you can check if without waiting for tomorrow

[groggily]

The old syntax extern IUP. will work as long as you have iup.dll in PATH. The new syntax use extern IUP; doesn't.

Error: Entry Point Not Found
The procedure entry point IUP could not be located in the dynamic link libary [My_ELENA_program.exe].

[arakov]

Oh, I see. I'll fix it asap

[arakov]

Please take a look. I've fixed it

[groggily]

It's working now. Thank you.

[groggily]

If the external function returns a string, how can I get it? If the external function returns a struct, how can I get and process it?

[arakov]

I'm implementing UnsafePointer template, so you can use it (I will do it asap). For strings the support already exists to some extent:

 pointer commandLinePtr := extern KERNEL32.GetCommandLineA();
 string cmdLine := commandLinePtr.Value;

 pointer commandLinePtrW := extern KERNEL32.GetCommandLineW();
 wide cmdLineW := commandLinePtr.Value;

But in case of a string, there may be problem with an encoding (string is UTF-8). I will fix it by providing method where you can specify the encoder.

Give me some time. And I'll let you know when it is ready. It will take me one, maximal two days

[groggily]

How to do type casting in ELENA?

[arakov]

The recommended way is

obj as:string

[groggily]

I found that cast is also highlighted as a keyword. How to do type casting with cast?

BTW, what are the differences between as and cast?

[arakov]

cast it is an original way how the typecasting is done in ELENA. The special cast message is sent to the object to invoke corresponding conversion routine. Conversion to the same type is done automatically.

:as is a inline template and is just more convenient way to do the same, the implementation is the same:

elena-lang/src60/system/operations/advanced/operations.l

Lines 35 to 36 in 20cf828

	public template as<T>(expr) : __included(statements)
	= cast T(expr);

[groggily]

Does ELENA use as as a keyword for type casting? If it doesn't, what do you think about this syntax extension?

use extern [Dll_name] as [My_preferred_name];

[groggily]

I put <dll key="LIBCXX">LIBC++</dll> into elc60.cfg and use extern LIBCXX; is working now.

[arakov]

Great, Still I will implement importing postfix attribute to do it without need to modify the project file

[arakov]

So, the support was added. It looks like this now:

use extern libcxx : importing("libc++");

[groggily]

Currently it is implemented in the project configuration file:

elena-lang/bin/elc60.config

Line 26 in 20cf828

<dll key="libc">libc.so.6</dll>

You said the project configuration file. But I think elc60.cfg and elc60.config are global compiler configuration files. There might be a misunderstanding here. Is it possible to put <dll key="LIBCXX">LIBC++</dll> into the project's .prj file?

[arakov]

Yes, they are. I have just two sets of them. One with local paths and one with global ones. (Local are in the the same folder as a compiler, the globals are in /etc/..), But local config files are only for Linux

Sure, You can place them in project file with the same path. In fact it is better, because the config files are changed from a version to version

[groggily]

Hi. I found that I can't use var and auto for the variable assigned with the return of an external function. It seems only int and pointer can serve as the type for the variable assigned with the return of an external function.

BTW, what are the differences between var and auto? They are both doing type inference to me.

[arakov]

There is some ambiguity here. var means that the variable has a weak type. It means you can assign any object, any stack-allocated object will be boxed before being assigned and so on.

var o := 2;  // both assignment is valid
o := "abc";

Please take a look at my post about how inefficient code can be with weak types - https://www.reddit.com/r/elena_lang/comments/1dvxla9/writing_efficient_code_in_elena/

auto in other way is similar to var in C# (in fact it is how C++ use it as well).

So you cannot assign the external function to a weak type, because the expected type is unknown and the boxing cannot be done.

And you cannot use auto as well for the same reason. Maybe later I will make possible to declare the external functions similar to C# and in that case auto variable can be used

[GB18030-2000]

As Elena can compile to native code, what do you think about making it possible to write a dll in Elena and use it in other languages like C? The language being garbage collected is not a problem, I think. I have seen dll written in Objective C.

[GB18030-2000]

If Objective C is not a good example, then modern .NET with native AOT can make a dll consumable by other languages, too:

https://github.com/bflattened/bflat/tree/master/samples/DynamicLibrary

[arakov]

Sure. I will think how it can be implemented. I will take a look at Objective C

[GB18030-2000]

@arakov This code no longer compile. What's going on?

// --- Program ---

import extensions;

use extern cxx : importing("libc++");

public program()
{
    var mangled := "_Z3fooi";
    int status;
    pointer demangled := cxx.__cxa_demangle(mangled, 0, 0, ref status);
    
    Console.printLine(demangled as:string);
}

error 147: Invalid attribute 'use'

[arakov]

Hi could you please provide a bit more information. The latest build for Linux works for me with your program:

Which version do you use? For what platform? Sometimes it is good to get a library version as well.

The following code can help:

import extensions;
import system'runtime;

public program()
{
  var info := Console.Package();

  Console.printLine(info.Name, " ", info.Version);
}

For me it was:

[GB18030-2000]

Latest Windows nightly (x64). Printed standard library 6.7.8

[arakov]

It is strange. I've just downloaded it - https://github.com/ELENA-LANG/elena-lang/releases/download/nightly/elena-lang-x64-nightly-6.7.2.zip and then have compiled you source, It works for me.

Can you show me your compiler output? Or at least the compiler version. Simply type elena64-cli. The output must be something like:

ELENA Command-line compiler 6.7.259 (C)2005-2025 by Aleksey Rakov, ELENA-LANG Org 

Please make sure that you have only one copy of the compiler executable (sometimes PATH may refer to outdated version).

Maybe someone else can try to compile this code and let us know.

[GB18030-2000]

Same output as you. I found that single file project will compile. But if I create a full project, it will not compile.

Ah. It seems if I keep the namespace as unnamed in Project Options, it will fail to compile.

[arakov]

It is a bit strange. But if it works for you, then fine.

The minimalistic project file can look like this:

<configuration>
    <files>
        <module>
            <file>source1.l</file>
        </module>
    </files>
    <project>
        <executable>test.exe</executable>
        <options></options>
        <namespace>sandbox</namespace>
        <template>console</template>
    </project>
</configuration>

The most important thing, is that the project has valid template - for simple console app - console.

The namespace name in this case is not important. But of course it is better to have more readable name.

Maybe something wrong with creating a new project from the scratch in IDE. If you encounter some strange behavior or inconvenience, please let me know.

[blearily]

Currently playing with GNUStep. Can you tell me why this code doesn't work? I ported it from Python:

// --- Program ---

import extensions;

use extern rt : importing("libobjc-4.6");

public program()
{
    pointer NSStringClass := rt.objc_getClass("NSString");
    pointer sel := rt.sel_registerName("stringWithUTF8String:");
    pointer str := rt.objc_msgSend(NSStringClass, sel, "Hello from runtime!");
    pointer logSel := rt.sel_registerName("NSLog:");
    pointer res := rt.objc_msgSend(rt.objc_getClass("NSLog"), logSel, str);
    
    //Console.printLine(res as:string);
}

[arakov]

Thank. I was able to compile it.

I see that the following code:

pointer NSStringClass := rt.objc_getClass("NSString");

returns nil.

By doc - https://developer.apple.com/documentation/objectivec/objc_getclass(_:) - it means the class NSString is not registered.
It looks like the objective-c library is not found. I will look further, maybe I can find out why.

[blearily]

Please let me know if you have any progress on this. Thank you.

[arakov]

Sure, I will let you know.

[arakov]

Could you send me a link to the example in Python to work with GNUStep?

[blearily]

It was an answer from an LLM and I didn't save it. I have also never tested to see if the Python code works. It seems the problem is with the Objective C runtime. If this is the case then there is nothing we can do to make it work. I'm sorry for wasting your time on this.

[GB18030-2000]

The C function returns a function pointer. How can I use this function pointer in ELENA?

pointer f := dll.get_func_pointer(foo, bar);

The function f has the following signature in C:

int f(struct B, const char*);

struct B is:

struct B {
    struct A *a;
    int b;
}

struct A is:

struct A {
    int x;
    int y;
}

[arakov]

Oh I see. It is not yet supported. I will implement this feature. It probably takes some time though. In ideal world I need to declare signature.

Currently for simplicity, the external function signature is not checked at all. Of course it is extremely unsafe, so I will fix it. And at that moment it will be possible to support a function pointer as well

[GB18030-2000]

I can make a wrapper to the C library to make it simpler. Instead of the big struct with function pointers, I will make several functions that return the function pointers. For example:

__declspec(dllexport) int (*foo(void))(int, int);

int bar(int a, int b) {
    return a + b;
}

int (*foo(void))(int, int) {
    return bar;
}

int result = foo()(10, 5);

[arakov]

Any way I have to support function pointer. It is not a big deal of course. But I need to think over possible solution from the language point of view. I'll do it, no worry.

[blearily]

If you can make a wrapper (making your own dll that wrap the original dll), why don't you make it simple enough to use the existing capacity of ELENA to call the external functions? Can you go away from function pointers? Your sample code only returns function pointers but not taking function pointers as parameters. This means it is only a clever trick to group related functions together. The current wrapper doesn't make sense (you no longer need grouping of related functions together, why are you still insist on using function pointers?).

[blearily]

Currently for simplicity, the external function signature is not checked at all. Of course it is extremely unsafe, so I will fix it. And at that moment it will be possible to support a function pointer as well

I think that is an advantage of ELENA. It will let me pass anything I want to the external functions without having to satisfy any type checkers. Of course, I don't write any mission critical apps in ELENA (I can't write mission critical apps in any languages, I'm an amateur).

[blearily]

I'm interested in passing C strings (const char*) and array of C strings (const char**) between ELENA and C.

p/s: Maybe array of array of C strings (const char***), too.

[arakov]

Though pointer does support simple arithmetic operation, so something like ptr + index * (:sizeof pointer) will work, I've just implemented a new classes : UnsafeArray and UnsafeArray<T>. So in this case your code may look like this:

public program()
{
   UnsafeArray ptr := dll.get_func_pointer();
   
   int i := 0;
   while (!ptr[i].isUnassigned()) {
      pointer item := ptr[i];
      
      Console.writeLine(item as:string);
      
      i++
   }
}

You will need a latest build for this as well (nightly build already has the latest code)

For two dimensional, is a bit more complicated. I think I could add support for this in the future as well. But now you may use
something like -

UnsafeArray<UnsafeArray> ptr := dll.get_func_pointer();

Let me know if you have other questions

[blearily]

Hi. I'm not the one that asked about function pointers. My problem is different. I want to get const char*/const char**/const char*** from an external C function and pass const char*/const char**/const char*** to an external C function.

[arakov]

Yes, the example above allow you to deal with char**. The code for char* / const char* is even simpler.

About passing data from ELENA side. You can pass any string as const char* (Though it is a bit tricky, because GC can change the pointers). The best way or course to pass stack allocated buffer when possible. I will add support for direct heap management later.

Similar with char**. It is possible but will require some efforts. So you declare an array of pointers. And assign each element with required string. Probably if you provide some code snippet I can suggest the solution.

[Curculigo]

How to deal with variadic functions like snprintf?

[arakov]

Not yet supported. I will look at it