Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding LXD tables #6249

Merged
merged 4 commits into from
Mar 16, 2020
Merged

Adding LXD tables #6249

merged 4 commits into from
Mar 16, 2020

Conversation

scoders-tob
Copy link
Contributor

@scoders-tob scoders-tob commented Feb 18, 2020

This is a collection of 9 LXD tables, which list information about LXD setup, images and containers.

Example queries:

osquery> lxd_
lxd_certificates      lxd_cluster           lxd_cluster_members   lxd_images            lxd_instance_config   lxd_instance_devices  lxd_instances         lxd_networks          lxd_storage_pools


osquery> select * from lxd_instances;
+--------+---------+----------+-----------+-------------------------------------+------------------------------------------------------------------+--------------+-------------+-----------------------------------------+------+-----------+
| name   | status  | stateful | ephemeral | created_at                          | base_image                                                       | architecture | os          | description                             | pid  | processes |
+--------+---------+----------+-----------+-------------------------------------+------------------------------------------------------------------+--------------+-------------+-----------------------------------------+------+-----------+
| second | Running | 0        | 0         | 2020-01-07T11:53:17.723230572-05:00 | 0931b693c877ef357b9e17b3195ae952a2450873923ffd2b34b60836ea730cfa | x86_64       | alpinelinux | Alpinelinux 3.8 x86_64 (20191230_13:00) | 2860 | 5         |
| first  | Stopped | 0        | 0         | 2020-01-08T12:20:01.654443235-05:00 | 979ff60086ca9fb1c93d3131f8eca291820a524f0bd0b7d621a3c2f5f41ef185 | x86_64       | ubuntu      | Ubuntu 18.04 LTS server (20200107)      | 0    | 0         |
| third  | Running | 0        | 0         | 2020-01-08T17:52:41.345852646-05:00 | 0931b693c877ef357b9e17b3195ae952a2450873923ffd2b34b60836ea730cfa | x86_64       | alpinelinux | Alpinelinux 3.8 x86_64 (20191230_13:00) | 3144 | 5         |
| cont1  | Running | 0        | 0         | 2020-01-21T12:30:06.20679995-05:00  | 0a10548b30ee85d9ac11c9e94cb0e2abaaef79a3b8af2405cc09f432b8631103 | x86_64       | alpinelinux | Alpinelinux 3.8 x86_64 (20200121_13:00) | 2655 | 5         |
+--------+---------+----------+-----------+-------------------------------------+------------------------------------------------------------------+--------------+-------------+-----------------------------------------+------+-----------+

osquery> select * from lxd_instance_devices where name = 'cont1';
+-------+--------+-------------+---------+---------+
| name  | device | device_type | key     | value   |
+-------+--------+-------------+---------+---------+
| cont1 | eth0   | nic         | name    | eth0    |
| cont1 | eth0   | nic         | nictype | bridged |
| cont1 | eth0   | nic         | parent  | lxdbr0  |
| cont1 | root   | disk        | path    | /       |
| cont1 | root   | disk        | pool    | default |
+-------+--------+-------------+---------+---------+

osquery> select * from lxd_instance_config where name = 'cont1';
+-------+---------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
| name  | key                       | value                                                                                                                                                       |
+-------+---------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
| cont1 | image.architecture        | amd64                                                                                                                                                       |
| cont1 | image.description         | Alpine 3.8 amd64 (20200121_13:00)                                                                                                                           |
| cont1 | image.os                  | Alpine                                                                                                                                                      |
| cont1 | image.release             | 3.8                                                                                                                                                         |
| cont1 | image.serial              | 20200121_13:00                                                                                                                                              |
| cont1 | image.type                | squashfs                                                                                                                                                    |
| cont1 | volatile.base_image       | 0a10548b30ee85d9ac11c9e94cb0e2abaaef79a3b8af2405cc09f432b8631103                                                                                            |
| cont1 | volatile.eth0.host_name   | veth670ed0fd                                                                                                                                                |
| cont1 | volatile.eth0.hwaddr      | 00:16:3e:12:88:4f                                                                                                                                           |
| cont1 | volatile.idmap.base       | 0                                                                                                                                                           |
| cont1 | volatile.idmap.current    | [{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}] |
| cont1 | volatile.idmap.next       | [{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}] |
| cont1 | volatile.last_state.idmap | [{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}] |
| cont1 | volatile.last_state.power | RUNNING                                                                                                                                                     |
+-------+---------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+

osquery> select * from lxd_images;
+------------------------------------------------------------------+--------------+--------+---------+---------------------------------------------+-------------------------+-----------------------------------------------+-----------+-------------+--------+--------+---------------------------+---------------------------+-------------------------------------+-------------------------------------+------------------------------------------+------------------------+---------------------------+---------------------+
| id                                                               | architecture | os     | release | description                                 | aliases                 | filename                                      | size      | auto_update | cached | public | created_at                | expires_at                | uploaded_at                         | last_used_at                        | update_source_server                     | update_source_protocol | update_source_certificate | update_source_alias |
+------------------------------------------------------------------+--------------+--------+---------+---------------------------------------------+-------------------------+-----------------------------------------------+-----------+-------------+--------+--------+---------------------------+---------------------------+-------------------------------------+-------------------------------------+------------------------------------------+------------------------+---------------------------+---------------------+
| 979ff60086ca9fb1c93d3131f8eca291820a524f0bd0b7d621a3c2f5f41ef185 | x86_64       | ubuntu | bionic  | ubuntu 18.04 LTS amd64 (release) (20200107) | old-ubuntu,old-ubuntu-1 | ubuntu-18.04-server-cloudimg-amd64-lxd.tar.xz | 187380496 | 0           | 0      | 0      | 2020-01-06T19:00:00-05:00 | 2023-04-25T20:00:00-04:00 | 2020-01-21T12:34:24.265420714-05:00 | 0001-01-01T00:00:00Z                | https://cloud-images.ubuntu.com/releases | simplestreams          |                           | 18.04               |
| d754399429a1b9d94ab026719efbf4ad6731e0704d769d3d142166521a5a22be | x86_64       | Alpine | 3.8     | Alpine 3.8 amd64 (20200128_18:16)           |                         | lxd.tar.xz                                    | 2011856   | 1           | 1      | 0      | 2020-01-27T19:00:00-05:00 | 1969-12-31T19:00:00-05:00 | 2020-01-28T14:19:34.770088844-05:00 | 2020-01-21T12:30:06.246604274-05:00 | https://images.linuxcontainers.org       | simplestreams          |                           | alpine/3.8          |
+------------------------------------------------------------------+--------------+--------+---------+---------------------------------------------+-------------------------+-----------------------------------------------+-----------+-------------+--------+--------+---------------------------+---------------------------+-------------------------------------+-------------------------------------+------------------------------------------+------------------------+---------------------------+---------------------+

osquery> select * from lxd_networks;
+-----------------+----------+---------+----------------+---------------------------+--------------------------+----------------+------------+------------------+--------------+-------------------+-------+-------+
| name            | type     | managed | ipv4_address   | ipv6_address              | used_by                  | bytes_received | bytes_sent | packets_received | packets_sent | hwaddr            | state | mtu   |
+-----------------+----------+---------+----------------+---------------------------+--------------------------+----------------+------------+------------------+--------------+-------------------+-------+-------+
| lxdbr0          | bridge   | 1       | 10.22.136.1/24 | fd42:bf24:45c1:8857::1/64 | cont1,first,second,third | 67612          | 240787     | 731              | 1373         | 0a:7e:06:e4:c1:a9 | up    | 1500  |
| lo              | loopback | 0       |                |                           |                          | 1118598        | 1118598    | 11675            | 11675        |                   | up    | 65536 |
| wlp0s20f3       | physical | 0       |                |                           |                          | 215180954      | 44486648   | 300032           | 169991       | 38:00:25:36:26:76 | up    | 1500  |
| br-5b606aec76cb | bridge   | 0       |                |                           |                          | 0              | 0          | 0                | 0            | 02:42:1c:cc:cc:33 | up    | 1500  |
| br-774f18b3f6bd | bridge   | 0       |                |                           |                          | 0              | 0          | 0                | 0            | 02:42:08:de:c5:c5 | up    | 1500  |
| docker0         | bridge   | 0       |                |                           |                          | 0              | 0          | 0                | 0            | 02:42:99:07:5b:43 | up    | 1500  |
+-----------------+----------+---------+----------------+---------------------------+--------------------------+----------------+------------+------------------+--------------+-------------------+-------+-------+

osquery> select * from lxd_cluster_members;
+-------------+-----------------+----------+---------+-----------------------------------------+
| server_name | url             | database | status  | message                                 |
+-------------+-----------------+----------+---------+-----------------------------------------+
| none        | https://0.0.0.0 | 0        | Offline | no heartbeat since 507h44m19.977819535s |
+-------------+-----------------+----------+---------+-----------------------------------------+

osquery> select * from lxd_cluster;
+-------------+---------+----------------------+--------------------+-------------------+---------------------+----------------------------------------------+
| server_name | enabled | member_config_entity | member_config_name | member_config_key | member_config_value | member_config_description                    |
+-------------+---------+----------------------+--------------------+-------------------+---------------------+----------------------------------------------+
|             | 0       | storage-pool         | default            | source            |                     | "source" property for storage pool "default" |
+-------------+---------+----------------------+--------------------+-------------------+---------------------+----------------------------------------------+

osquery> select * from lxd_storage_pools;
+---------+--------+------------------------------------------------+------+--------------+--------------+-------------+--------------+
| name    | driver | source                                         | size | space_used   | space_total  | inodes_used | inodes_total |
+---------+--------+------------------------------------------------+------+--------------+--------------+-------------+--------------+
| default | dir    | /var/snap/lxd/common/lxd/storage-pools/default |      | 153647570944 | 294216040448 | 2043458     | 18317312     |
+---------+--------+------------------------------------------------+------+--------------+--------------+-------------+--------------+

osquery> select* from lxd_certificates;
+-----------+--------+------------------------------------------------------------------+---------------------------------------------------------------+
| name      | type   | fingerprint                                                      | certificate      												|
+-----------+--------+------------------------------------------------------------------+---------------------------------------------------------------+
| 127.0.0.1 | client | e93b07737f24daa368ecb55e12cfef351d84380a0e88edce7b205c23368d0ebf | -----BEGIN CERTIFICATE----- xxxx -----END CERTIFICATE----- 	|
+-----------+--------+------------------------------------------------------------------+---------------------------------------------------------------+


@lgtm-com
Copy link

lgtm-com bot commented Feb 24, 2020

LGTM pull request analysis was skipped for b5227f8 by Smjert. Analysis of future commits will happen as normal.

Copy link
Member

@theopolis theopolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

The only nitpick I have is moving the Statuses to the newer APIs using ::success() and ::failure().

Also, just a note that ptree reading/parsing is quite slow compared to RapidJSON. If folks find they need this information faster (save some memory and CPU), consider refactoring. The RapidJSON and our JSON wrapper APIs are a bit more confusing and prone to error compared to ptree so let's only do that if we need it.

local::stream_protocol::endpoint ep(FLAGS_lxd_socket);
local::stream_protocol::iostream stream(ep);
if (!stream) {
return Status(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update these to the new Status::failure pattern?

@Smjert
Copy link
Member

Smjert commented Feb 28, 2020

Also, just a note that ptree reading/parsing is quite slow compared to RapidJSON. If folks find they need this information faster (save some memory and CPU), consider refactoring. The RapidJSON and our JSON wrapper APIs are a bit more confusing and prone to error compared to ptree so let's only do that if we need it.

This has been loosely based on the dockerApi implementation.
I know that our objective is to remove boost::ptree but I think that we also have to provide better API first, especially in the form of checking document format validity and type conversions.

@theopolis theopolis merged commit 4e5dcd9 into osquery:master Mar 16, 2020
@Smjert Smjert deleted the lxd branch March 18, 2020 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants