annotate plugins/mod_offline.lua @ 11523:5f15ab7c6ae5

Statistics: Rewrite statistics backends to use OpenMetrics The metric subsystem of Prosody has had some shortcomings from the perspective of the current state-of-the-art in metric observability. The OpenMetrics standard [0] is a formalization of the data model (and serialization format) of the well-known and widely-used Prometheus [1] software stack. The previous stats subsystem of Prosody did not map well to that format (see e.g. [2] and [3]); the key reason is that it was trying to do too much math on its own ([2]) while lacking first-class support for "families" of metrics ([3]) and structured metric metadata (despite the `extra` argument to metrics, there was no standard way of representing common things like "tags" or "labels"). Even though OpenMetrics has grown from the Prometheus world of monitoring, it maps well to other popular monitoring stacks such as: - InfluxDB (labels can be mapped to tags and fields as necessary) - Carbon/Graphite (labels can be attached to the metric name with dot-separation) - StatsD (see graphite when assuming that graphite is used as backend, which is the default) The util.statsd module has been ported to use the OpenMetrics model as a proof of concept. An implementation which exposes the util.statistics backend data as Prometheus metrics is ready for publishing in prosody-modules (most likely as mod_openmetrics_prometheus to avoid breaking existing 0.11 deployments). At the same time, the previous measure()-based API had one major advantage: It is really simple and easy to use without requiring lots of knowledge about OpenMetrics or similar concepts. For that reason as well as compatibility with existing code, it is preserved and may even be extended in the future. However, code relying on the `stats-updated` event as well as `get_stats` from `statsmanager` will break because the data model has changed completely; in case of `stats-updated`, the code will simply not run (as the event was renamed in order to avoid conflicts); the `get_stats` function has been removed completely (so it will cause a traceback when it is attempted to be used). Note that the measure_*_event methods have been removed from the module API. I was unable to find any uses or documentation and thus deemed they should not be ported. Re-implementation is possible when necessary. [0]: https://openmetrics.io/ [1]: https://prometheus.io/ [2]: #959 [3]: #960
author Jonas Schäfer <jonas@wielicki.name>
date Sun, 18 Apr 2021 11:47:41 +0200
parents 7072569044d4
children 74b9e05af71e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2680
1b267876246a Backed out changeset e7b09424b486 (not removing mod_offline for trunk)
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
1 -- Prosody IM
1b267876246a Backed out changeset e7b09424b486 (not removing mod_offline for trunk)
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
2 -- Copyright (C) 2008-2009 Matthew Wild
1b267876246a Backed out changeset e7b09424b486 (not removing mod_offline for trunk)
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
3 -- Copyright (C) 2008-2009 Waqas Hussain
5776
bd0ff8ae98a8 Remove all trailing whitespace
Florian Zeitz <florob@babelmonkeys.de>
parents: 3973
diff changeset
4 --
2680
1b267876246a Backed out changeset e7b09424b486 (not removing mod_offline for trunk)
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
5 -- This project is MIT/X11 licensed. Please see the
1b267876246a Backed out changeset e7b09424b486 (not removing mod_offline for trunk)
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
6 -- COPYING file in the source package for more information.
1b267876246a Backed out changeset e7b09424b486 (not removing mod_offline for trunk)
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
7 --
1b267876246a Backed out changeset e7b09424b486 (not removing mod_offline for trunk)
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
8
2737
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
9
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
10 local datetime = require "util.datetime";
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
11 local jid_split = require "util.jid".split;
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
12
8024
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
13 local offline_messages = module:open_store("offline", "archive");
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
14
2737
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
15 module:add_feature("msgoffline");
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
16
3972
a05cf5d9c7ab mod_message, mod_offline: Change message/offline/store -> message/offline/handle
Robert Hoelz <rob@hoelz.ro>
parents: 3971
diff changeset
17 module:hook("message/offline/handle", function(event)
2737
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
18 local origin, stanza = event.origin, event.stanza;
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
19 local to = stanza.attr.to;
8024
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
20 local node;
2737
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
21 if to then
8024
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
22 node = jid_split(to)
2737
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
23 else
8024
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
24 node = origin.username;
2737
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
25 end
5776
bd0ff8ae98a8 Remove all trailing whitespace
Florian Zeitz <florob@babelmonkeys.de>
parents: 3973
diff changeset
26
10296
7072569044d4 mod_offline: Log a debug message when message is stored
Kim Alvefur <zash@zash.se>
parents: 10242
diff changeset
27 local ok = offline_messages:append(node, nil, stanza, os.time(), "");
7072569044d4 mod_offline: Log a debug message when message is stored
Kim Alvefur <zash@zash.se>
parents: 10242
diff changeset
28 if ok then
7072569044d4 mod_offline: Log a debug message when message is stored
Kim Alvefur <zash@zash.se>
parents: 10242
diff changeset
29 module:log("debug", "Saved to offline storage: %s", stanza:top_tag());
7072569044d4 mod_offline: Log a debug message when message is stored
Kim Alvefur <zash@zash.se>
parents: 10242
diff changeset
30 end
7072569044d4 mod_offline: Log a debug message when message is stored
Kim Alvefur <zash@zash.se>
parents: 10242
diff changeset
31 return ok;
6834
750a97b45f88 mod_offline: Decrease priority of event handlers so 3rd party hooks fire first by default
Kim Alvefur <zash@zash.se>
parents: 5776
diff changeset
32 end, -1);
2737
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
33
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
34 module:hook("message/offline/broadcast", function(event)
2251f8898900 mod_offline: Convert to Unix line endings.
Waqas Hussain <waqas20@gmail.com>
parents: 2680
diff changeset
35 local origin = event.origin;
10242
7e5d2a6c9390 mod_offline: Add some debug logging to reduce confusion
Kim Alvefur <zash@zash.se>
parents: 8030
diff changeset
36 origin.log("debug", "Broadcasting offline messages");
3520
dc94193ec942 Alter mod_offline to respect negative priority resources
Robert Hoelz <rob@hoelz.ro>
parents: 2737
diff changeset
37
3973
f5416372350d Don't check priority in mod_offline
Robert Hoelz <rob@hoelz.ro>
parents: 3972
diff changeset
38 local node, host = origin.username, origin.host;
3540
bc139431830b Monster whitespace commit (beware the whitespace monster).
Waqas Hussain <waqas20@gmail.com>
parents: 3520
diff changeset
39
8024
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
40 local data = offline_messages:find(node);
3973
f5416372350d Don't check priority in mod_offline
Robert Hoelz <rob@hoelz.ro>
parents: 3972
diff changeset
41 if not data then return true; end
8024
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
42 for _, stanza, when in data do
8eec715c13a6 mod_offline: Switch to using archive store via the storagemanager API
Kim Alvefur <zash@zash.se>
parents: 6834
diff changeset
43 stanza:tag("delay", {xmlns = "urn:xmpp:delay", from = host, stamp = datetime.datetime(when)}):up(); -- XEP-0203
3973
f5416372350d Don't check priority in mod_offline
Robert Hoelz <rob@hoelz.ro>
parents: 3972
diff changeset
44 origin.send(stanza);
3540
bc139431830b Monster whitespace commit (beware the whitespace monster).
Waqas Hussain <waqas20@gmail.com>
parents: 3520
diff changeset
45 end
10242
7e5d2a6c9390 mod_offline: Add some debug logging to reduce confusion
Kim Alvefur <zash@zash.se>
parents: 8030
diff changeset
46 local ok = offline_messages:delete(node);
7e5d2a6c9390 mod_offline: Add some debug logging to reduce confusion
Kim Alvefur <zash@zash.se>
parents: 8030
diff changeset
47 if type(ok) == "number" and ok > 0 then
7e5d2a6c9390 mod_offline: Add some debug logging to reduce confusion
Kim Alvefur <zash@zash.se>
parents: 8030
diff changeset
48 origin.log("debug", "%d offline messages consumed");
7e5d2a6c9390 mod_offline: Add some debug logging to reduce confusion
Kim Alvefur <zash@zash.se>
parents: 8030
diff changeset
49 end
3973
f5416372350d Don't check priority in mod_offline
Robert Hoelz <rob@hoelz.ro>
parents: 3972
diff changeset
50 return true;
6834
750a97b45f88 mod_offline: Decrease priority of event handlers so 3rd party hooks fire first by default
Kim Alvefur <zash@zash.se>
parents: 5776
diff changeset
51 end, -1);