在Elixir或Erlang中,如何在运行时动态地创建和加载模块?

pgx2nnw8  于 2022-12-08  发布在  Erlang
关注(0)|答案(3)|浏览(190)

The basic scenario is this: I need to load text from a database, and then turn that text into an Elixir module (or an Erlang module) and then make calls into it. The text is effectively the same as a module file. So this is a form of hot code loading. I want to compile the "file" and then load the resulting module, then make calls into it. Later I'll unload it. The only difference is the code exists in a database rather than a file on the disk. (and it doesn't exist at the time that I'm writing the code that will be loading it.)
I know Erlang supports hot code loading, but seems focused on compiling files on disk and then loading the beams. I wish to do this as a more dynamic process, and I won't be replacing running code, but loading the code, then running it, then unloading it.
There are several facilities in Elixir for evaluating code at runtime. I'm trying to figure out how to do this with them, and the documentation is a bit sparse.

Code.compile_string(string, "nofile")

"returns a list of tuples where the first element is the module name and the second one is its binary". So, now I have the modules names and their binaries, but I do not know of a way to then load the binaries into the runtime and call into them. How would I do that? (There's no function for that in the Code library that I can see.)
Possibly I could then use the Erlang function:

:code.load_binary(Module, Filename, Binary)  ->
           {module, Module} | {error, What}

Ok, so this returns a tuple with the atom "module" and then the Module. If the string loaded from the database defined a module called "Paris", how in my code would I then execute

paris.handler([parameters])

since I don't know in advance that there will be a module called paris? I could know, by having the string "paris" also stored in the database that this is the name, but is there any way of calling into a module, using a string as the name of the module you're calling?
There is also:

eval(string, binding // [], opts // [])

Which evaluates the contents of the string. Can this string be the entire definition of a module? It appears not. I'd like to be able to write this code that's being evaluated in such a way that it has multiple functions that call each other--e.g. a complete little program, with a pre-defined entry point (Which could be a main, such as "DynamicModule.handle([parameter, list])"
Then there's the EEx module, which has:

compile_string(source, options // [])

Which is great for doing templates. But ultimately it only seems to work for the use case where there's a string and you've got Elixir code embedded in it. It evaluates the string in the context of the options and produces a string. I'm seeking to compile the string into one or more functions that I can then call. (If I can only make one function that's fine, that function can pattern match or switch into doing the other things that are needed....)
I know this is unconventional, but I have my reasons for doing it this way and they are good ones. I'm looking for advice about how to do this, but don't need to be told "don't do that". It seems like it should be possible, Erlang supports hot code loading and Elixir is pretty dynamic, but I just don't know the syntax, or the right functions. I'll be monitoring this question closely. Thanks in advance!

EDITS based on the first answers:

Thanks for the answers, this is good progress. As Yuri showed, eval can define a module, and as José points out, I can just use code eval for small parts of code with bindings.
The code being evaluated (whether turned into a module, or not) is going to be fairly complex. And its development would be best involving breaking it down into functions and calling those functions.
To help, let me provide some context. Assume I'm building a web framework. The code loaded in from the database is handlers for specific URIs. So, when an HTTP call comes in, I might load the code for example.com/blog/ This page might involve several different things, such as comments, a list of recent posts, etc.
Since many people are hitting the page at the same time, I'm spawning a process to handle each page view. Thus there are many times when this code may be evaluated simultaneously, for different requests.
The the module solution allows one to break the code up into functions for different parts of the page (eg: the list of posts, comments, etc. ) And I would load the module once, at startup, and let many processes spawn that call into it. The module is global, correct?
What happens if there's a module already defined? EG: When the module changes, and there are processes already calling that module.
In iex, I am able to redefine a module that's already been defined:

iex(20)> Code.eval "defmodule A do\ndef a do\n5\nend\nend"
nofile:1: redefining module A

What happens if I redefine the module at runtime, to all the processes currently calling into that module? Also, will this redefinition work outside of iex, in normal operation?
Assuming that redefining the module would be problematic, and that modules being global might run into problems with namespace collisions, I looked into using eval to define a function.

如果我可以只让数据库中的代码定义函数,那么这些函数就在任何进程的范围内,我们就不可能发生全局冲突。
然而,这似乎行不通:

iex(31)> q = "f = function do
...(31)> x, y when x > 0 -> x+y
...(31)> x, y -> x* y
...(31)> end"
"f = function do\nx, y when x > 0 -> x+y\nx, y -> x* y\nend"
iex(32)> Code.eval q
{#Fun<erl_eval.12.82930912>,[f: #Fun<erl_eval.12.82930912>]}
iex(33)> f
** (UndefinedFunctionError) undefined function: IEx.Helpers.f/0
    IEx.Helpers.f()
    erl_eval.erl:572: :erl_eval.do_apply/6
    src/elixir.erl:110: :elixir.eval_forms/3
    /Users/jose/Work/github/elixir/lib/iex/lib/iex/loop.ex:18: IEx.Loop.do_loop/1

iex(33)> f.(1,3)
** (UndefinedFunctionError) undefined function: IEx.Helpers.f/0
    IEx.Helpers.f()
    erl_eval.erl:572: :erl_eval.do_apply/6
    erl_eval.erl:355: :erl_eval.expr/5
    src/elixir.erl:110: :elixir.eval_forms/3
    /Users/jose/Work/github/elixir/lib/iex/lib/iex/loop.ex:18: IEx.Loop.do_loop/1

我也试探着:

iex(17)> y = Code.eval "fn(a,b) -> a + b end"
{#Fun<erl_eval.12.82930912>,[]}
iex(18)> y.(1,2)
** (BadFunctionError) bad function: {#Fun<erl_eval.12.82930912>,[]}
    erl_eval.erl:559: :erl_eval.do_apply/5
    src/elixir.erl:110: :elixir.eval_forms/3
    /Users/jose/Work/github/elixir/lib/iex/lib/iex/loop.ex:18: IEx.Loop.do_loop/1

所以,总结一下:
1.当有进程调用模块时,是否可以使用Code.eval重新定义模块?
1.是否可以使用Code.eval来创建函数,使其作用域绑定到调用Code.eval的进程?
1.如果你明白我想做什么,你能提出一个更好的方法吗?
另外,如果有更好的论坛,我应该问这个问题,请随时告诉我。如果有文档或相关的例子,我应该阅读,请随时告诉我。我不是想让你做所有的工作,我只是不能弄清楚自己。
我正在学习Elixir,特别是动态评估代码的能力,但我的Elixir知识现在是最少的-我刚刚开始-我的Erlang也生 rust 了。
非常感谢!

ee7vknir

ee7vknir1#

As you described, there are many different approaches you could take by ultimately they boil down to two different categories: 1) code compilation and 2) code evaluation. The example you described above requires compilation, which will define a module and then you would have to invoke it. However, as you found out, it requires defining a module name and potentially purging and discarding those modules when the database changes. Also, note that, defining modules may exhaust the atom table, as an atom is created for every module. I would only use this approach if you need to compile at maximum a dozen modules.
(A small note, Code.compile_string already defines the module, so you don't need to call load_binary ).
Maybe a simpler approach is to call Code.eval passing the code from the database, thus code evaluation. It works fine if all you want to is to evaluate some custom rules. The Code.eval accepts a binding, which would allow you to pass parameters to the code. Let's suppose you have "a + b" stored in the database, where a and b are the parameters, you could evaluate it as:

Code.eval "a + b", [a: 1, b: 1]

EDIT BASED ON QUESTION'S EDITS

If you are evaluating code, keep in mind that Code.eval returns the result of evaluating the code and the new binding, so your example above would be better written as:

q = "f = function do
x, y when x > 0 -> x+y
x, y -> x* y
end"

{ _, binding } = Code.eval q
binding[:f].(1, 2)

However, given your use case, I wouldn't consider using eval but I would indeed compile a module. Since the information is coming from the database, you can use this fact to generate unique modules per record for you. For example, you can assume developers will be adding the contents of a module to the database, something like:

def foo, do: 1
def bar, do: 1

After you get this information from the database, you can compile it using:

record   = get_record_from_database
id       = record.id
contents = Code.string_to_quoted!(record.body)
module   = Module.concat(FromDB, "Data#{record.id}")
Module.create module, contents, Macro.Env.location(__ENV__)
module

Where record is anything that you get back from the database. Assuming the record id is 1 , it would define a module FromDB.Data1 which then you would be able to invoke foo and bar .
Regarding code reloading, both defmodule and Module.create use :code.load_binary to load the module. Which means that if you update the module, the old version is still kept around until another new version is loaded.
One of the things you should add as well is cache expiration, so you don't need to compile a module on every request. This can be done if you have a lock_version in the database that you increment every time you change the record contents. The final code would be something like:

record  = get_record_from_database
module  = Module.concat(FromDB, "Data#{record.id}")
compile = :code.is_loaded(module) == false or
            record.lock_version > module.lock_version

if compile do
  id       = record.id
  contents = Code.string_to_quoted!(record.body)
  contents = quote do
    def lock_version, do: unquote(record.lock_version)
    unquote(contents)
  end
  Module.create module, contents, Macro.Env.location(__ENV__)
end

module
aelbi1ox

aelbi1ox2#

Code.eval可用于定义模块:

iex(1)> Code.eval "defmodule A do\ndef a do\n1\nend\nend" 
{{:module,A,<<70,79,82,49,0,0,2,168,66,69,65,77,65,116,111,109,0,0,0,98,0,0,0,11,8,69,108,105,120,105,114,45,65,8,95,95,105,110,102,111,95,95,4,100,111,99,115,9,102,117,...>>,{:a,0}},[]}
iex(2)> A.a
1

这有帮助吗?

o75abkj4

o75abkj43#

您是否检查过:Dynamic Compile Library by Jacob Vorreuter。请参阅下面的示例

1> String = "-module(add).\n -export([add/2]). \n add(A,B) -> A + B. \n".
"-module(add).\n -export([add/2]). \n add(A,B) -> A + B. \n"
2> dynamic_compile:load_from_string(String).
{module,add}
3> add:add(2,5).
7
4>

此外,请查看此question及其answer

相关问题