Tests Report for pkg-system-tests

Date: 2022-18-05 16:38

Jenkins job: https://jenkins.aws.isc.org/job/kea-dev/job/pkg-system-tests

Analyzed last 86 builds

Last build: 460, 2022-05-13 08:43 (5 days ago)

Failures History

Total & Passed History

Regressions

No Test Name Instability
last 5, previous 20 (6-25), earlier builds
Comments
1 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[postgresql-parameter0] 3/5 5/11
2 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[postgresql-parameter1] 4/5 4/11

Failures

No Test Name Last Build 460 Result Last Success Build Instability
last 5, previous 20 (6-25), earlier builds
Comments
1 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[memfile-parameter0] 460 latest 457 1/5 3/11
2 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[memfile-parameter1] 460 latest 457 2/5 0/11
3 run tests-fedora-34-v4-tests.dhcp.hooks.leases_cmds.test_leases_cmds.test_v4_lease_cmds_lease_get_negative 460 latest 0/5 0/12
4 run tests-fedora-34-v4-tests.dhcp.hooks.test_subnet_cmds.test_hook_v4_subnet_grow_cb_command[http] 460 latest 0/5 0/12
5 run tests-fedora-34-v4-tests.dhcp.hooks.test_subnet_cmds.test_hook_v4_subnet_grow_cb_command[socket] 460 latest 0/5 0/12
6 run tests-fedora-34-v4-tests.dhcp.test_backend_migration.test_v4_lease_upload_duplicate[mysql] 460 latest 0/5 0/20 0/7
7 run tests-fedora-34-v4-tests.dhcp.test_backend_migration.test_v4_lease_upload_duplicate[postgresql] 460 latest 0/5 0/20 0/7
8 run tests-fedora-34-v4-tests.dhcp.test_logging.test_v4_loggers_all_different_levels_different_file 460 latest 0/5 0/12
9 run tests-fedora-34-v4-tests.dhcp.v4.test_lease_cache.test_lease_cache_disabled[parameter1-memfile] 460 latest 0/5 0/12
10 run tests-fedora-34-v4-tests.dhcp.v4.test_lease_cache.test_lease_cache_disabled[parameter1-mysql] 460 latest 0/5 0/12
11 run tests-fedora-34-v4-tests.dhcp.v4.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql] 460 latest 0/5 0/12
12 run tests-fedora-34-v6-tests.dhcp.hooks.test_subnet_cmds.test_hook_v6_subnet_grow_cb_command[http] 460 latest 0/5 0/12
13 run tests-fedora-34-v6-tests.dhcp.hooks.test_subnet_cmds.test_hook_v6_subnet_grow_cb_command[socket] 460 latest 0/5 0/12
14 run tests-fedora-34-v6-tests.dhcp.test_backend_migration.test_v6_lease_upload_duplicate[mysql] 460 latest 0/5 0/20 0/7
15 run tests-fedora-34-v6-tests.dhcp.test_backend_migration.test_v6_lease_upload_duplicate[postgresql] 460 latest 0/5 0/20 0/7
16 run tests-fedora-34-v6-tests.dhcp.test_logging.test_v6_loggers_all_different_levels_different_file 460 latest 0/5 0/12
17 run tests-fedora-34-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_disabled[parameter1-memfile] 460 latest 0/5 0/12
18 run tests-fedora-34-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_disabled[parameter1-mysql] 460 latest 0/5 0/12
19 run tests-fedora-34-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql] 460 latest 0/5 0/12
20 run tests-ubuntu-20.04-ddns-tests.ddns.test_ddns_gss_tsig.test_ddns4_gss_tsig_complex_scenario[system_domain1] 460 latest 0/5 0/11 wlodek
21 run tests-ubuntu-20.04-ddns-tests.ddns.test_ddns_gss_tsig.test_ddns4_gss_tsig_complex_scenario[system_domain2] 460 latest 0/5 0/11 wlodek
22 run tests-ubuntu-20.04-ddns-tests.ddns.test_ddns_gss_tsig.test_ddns_gss_tsig_manual_expiration[system_and_domain1] 460 latest 0/5 0/11 wlodek
23 run tests-ubuntu-20.04-ddns-tests.ddns.test_ddns_gss_tsig.test_ddns_gss_tsig_manual_expiration[system_and_domain2] 460 latest 0/5 0/11 wlodek
24 run tests-ubuntu-20.04-v4-tests.dhcp.hooks.leases_cmds.test_leases_cmds.test_v4_lease_cmds_lease_get_negative 460 latest 0/5 0/11
25 run tests-ubuntu-20.04-v4-tests.dhcp.hooks.test_subnet_cmds.test_hook_v4_subnet_grow_cb_command[http] 460 latest 0/5 0/11
26 run tests-ubuntu-20.04-v4-tests.dhcp.hooks.test_subnet_cmds.test_hook_v4_subnet_grow_cb_command[socket] 460 latest 0/5 0/11
27 run tests-ubuntu-20.04-v4-tests.dhcp.test_backend_migration.test_v4_lease_dump[mysql] 460 latest 0/5 0/20 0/6
28 run tests-ubuntu-20.04-v4-tests.dhcp.test_backend_migration.test_v4_lease_dump[postgresql] 460 latest 0/5 0/20 0/6
29 run tests-ubuntu-20.04-v4-tests.dhcp.test_backend_migration.test_v4_lease_upload_duplicate[mysql] 460 latest 0/5 0/20 0/6
30 run tests-ubuntu-20.04-v4-tests.dhcp.test_backend_migration.test_v4_lease_upload_duplicate[postgresql] 460 latest 0/5 0/20 0/6
31 run tests-ubuntu-20.04-v4-tests.dhcp.test_logging.test_v4_loggers_all_different_levels_different_file 460 latest 0/5 0/11
32 run tests-ubuntu-20.04-v4-tests.dhcp.v4.test_lease_cache.test_lease_cache_disabled[parameter1-memfile] 460 latest 0/5 0/11
33 run tests-ubuntu-20.04-v6-tests.dhcp.hooks.test_subnet_cmds.test_hook_v6_subnet_grow_cb_command[http] 460 latest 0/5 0/11
34 run tests-ubuntu-20.04-v6-tests.dhcp.hooks.test_subnet_cmds.test_hook_v6_subnet_grow_cb_command[socket] 460 latest 0/5 0/11
35 run tests-ubuntu-20.04-v6-tests.dhcp.test_backend_migration.test_v6_lease_dump[mysql] 460 latest 0/5 0/20 0/6
36 run tests-ubuntu-20.04-v6-tests.dhcp.test_backend_migration.test_v6_lease_dump[postgresql] 460 latest 0/5 0/20 0/6
37 run tests-ubuntu-20.04-v6-tests.dhcp.test_backend_migration.test_v6_lease_upload_duplicate[mysql] 460 latest 0/5 0/20 0/6
38 run tests-ubuntu-20.04-v6-tests.dhcp.test_backend_migration.test_v6_lease_upload_duplicate[postgresql] 460 latest 0/5 0/20 0/6
39 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[postgresql-parameter0] 460 latest 459 3/5 5/11
40 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[postgresql-parameter1] 460 latest 459 4/5 4/11
41 run tests-fedora-34-cb-tests.config_backend.test_cb_v4_server_tag.test_server_tag_global_option4[mysql] 460 latest 0/5 0/12 kea #1600
42 run tests-fedora-34-cb-tests.config_backend.test_cb_v4_server_tag.test_server_tag_global_option4[postgresql] 460 latest 0/5 0/12 kea #1600
43 run tests-fedora-34-cb-tests.config_backend.test_cb_v6_server_tag.test_server_tag_global_option[mysql] 460 latest 0/5 0/12 kea #1600
44 run tests-fedora-34-cb-tests.config_backend.test_cb_v6_server_tag.test_server_tag_global_option[postgresql] 460 latest 0/5 0/12 kea #1600
45 run tests-fedora-34-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vendor_encapsulated_siemens_defined_in_class 460 latest 0/5 0/12 kea #1683
46 run tests-fedora-34-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vivso_suboptions_siemens_defined_in_class 460 latest 0/5 0/12 kea #1683
47 run tests-ubuntu-20.04-cb-tests.config_backend.test_cb_v4_server_tag.test_server_tag_global_option4[mysql] 460 latest 0/5 0/11 kea #1600
48 run tests-ubuntu-20.04-cb-tests.config_backend.test_cb_v4_server_tag.test_server_tag_global_option4[postgresql] 460 latest 0/5 0/11 kea #1600
49 run tests-ubuntu-20.04-cb-tests.config_backend.test_cb_v6_server_tag.test_server_tag_global_option[mysql] 460 latest 0/5 0/11 kea #1600
50 run tests-ubuntu-20.04-cb-tests.config_backend.test_cb_v6_server_tag.test_server_tag_global_option[postgresql] 460 latest 0/5 0/11 kea #1600
51 run tests-ubuntu-20.04-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vendor_encapsulated_siemens_defined_in_class 460 latest 0/5 0/11 kea #1683
52 run tests-ubuntu-20.04-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vivso_suboptions_siemens_defined_in_class 460 latest 0/5 0/11 kea #1683
53 run tests-ubuntu-20.04-v4-tests.dhcp.v4.test_lease_cache.test_lease_cache_disabled[parameter1-mysql] 460 latest 0/5 0/11 kea #1796
54 run tests-ubuntu-20.04-v4-tests.dhcp.v4.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql] 460 latest 0/5 0/11 kea #1796
55 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_disabled[parameter1-memfile] 460 latest 0/5 0/11 kea #1796
56 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_disabled[parameter1-mysql] 460 latest 0/5 0/11 kea #1796
57 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql] 460 latest 0/5 0/11 kea #1796

Fixes

No Test Name Instability
last 5, previous 20 (6-25), earlier builds
Comments
1 run tests-fedora-34-ddns-tests.ddns.test_ddns6.test_ddns6_notsig_rev_Nflag_override_no_update 1/5 0/12
2 run tests-fedora-34-ddns-tests.ddns.test_ddns6.test_ddns6_notsig_rev_success_withoutflag 1/5 0/12
3 run tests-fedora-34-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vendor_encapsulated_options_space_siemens 1/5 0/12
4 run tests-fedora-34-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[postgresql-parameter1] 2/5 2/12
5 run tests-ubuntu-20.04-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vendor_encapsulated_options_space_siemens 1/5 0/11
6 run tests-ubuntu-20.04-v4-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v4-False] 1/5 0/11
7 run tests-ubuntu-20.04-v4-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v4-True] 1/5 0/11
8 run tests-ubuntu-20.04-v6-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v6-False] 1/5 0/11
9 run tests-ubuntu-20.04-v6-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v6-True] 1/5 0/11

Passes

No Test Name Last Build 460 Result Last Fail Build Instability
last 5, previous 20 (6-25), earlier builds
Comments
1 run tests-fedora-34-ddns-tests.ddns.test_ddns6.test_ddns6_notsig_rev_Nflag_override_no_update 460 latest 459 1/5 0/12
2 run tests-fedora-34-ddns-tests.ddns.test_ddns6.test_ddns6_notsig_rev_success_withoutflag 460 latest 459 1/5 0/12
3 run tests-fedora-34-ha-tests.HA.test_sync.test_HA_hot_standby_different_page_size_sync[v4_bootp-alphabetical-memfile] 460 latest 434 0/5 0/20 2/17
4 run tests-fedora-34-ha-tests.HA.test_sync.test_HA_hot_standby_different_page_size_sync[v4_bootp-alphabetical-mysql] 460 latest 434 0/5 0/20 2/17
5 run tests-fedora-34-ha-tests.HA.test_sync.test_HA_hot_standby_different_page_size_sync[v4_bootp-alphabetical-postgresql] 460 latest 434 0/5 0/20 2/17
6 run tests-fedora-34-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vendor_encapsulated_options_space_siemens 460 latest 459 1/5 0/12
7 run tests-fedora-34-v6-tests.dhcp.hooks.test_radius.test_radius_framed_pool[v6-single-attribute] 460 latest 455 1/5 0/12
8 run tests-fedora-34-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[memfile-parameter1] 460 latest 458 2/5 0/12
9 run tests-fedora-34-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[postgresql-parameter0] 460 latest 453 0/5 5/12
10 run tests-fedora-34-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[postgresql-parameter1] 460 latest 459 2/5 2/12
11 run tests-ubuntu-20.04-ha-tests.HA.test_sync.test_HA_hot_standby_different_page_size_sync[v4_bootp-alphabetical-memfile] 460 latest 434 0/5 1/20 1/16
12 run tests-ubuntu-20.04-ha-tests.HA.test_sync.test_HA_hot_standby_different_page_size_sync[v4_bootp-alphabetical-mysql] 460 latest 434 0/5 1/20 1/16
13 run tests-ubuntu-20.04-ha-tests.HA.test_sync.test_HA_hot_standby_different_page_size_sync[v4_bootp-alphabetical-postgresql] 460 latest 434 0/5 1/20 1/16
14 run tests-ubuntu-20.04-v4-tests.dhcp.protocol.test_vendor_options.test_v4_options_vendor_encapsulated_options_space_siemens 460 latest 459 1/5 0/11
15 run tests-ubuntu-20.04-v4-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v4-False] 460 latest 459 1/5 0/11
16 run tests-ubuntu-20.04-v4-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v4-True] 460 latest 459 1/5 0/11
17 run tests-ubuntu-20.04-v6-tests.dhcp.hooks.test_radius.test_radius_framed_pool[v6-single-attribute] 460 latest 455 1/5 0/11
18 run tests-ubuntu-20.04-v6-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v6-False] 460 latest 459 1/5 0/11
19 run tests-ubuntu-20.04-v6-tests.dhcp.tls.test_ca_tls.test_ca_tls_basic[v6-True] 460 latest 459 1/5 0/11
20 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[mysql-parameter0] 460 latest 458 4/5 3/11
21 run tests-ubuntu-20.04-v6-tests.dhcp.v6.test_lease_cache.test_lease_cache_enabled[mysql-parameter1] 460 latest 458 4/5 3/11

Tests that are mentioned in the spreadsheet, but don't exist or are not failing:
51: :
52: run tests-fedora-34-v4-tests.dhcpv4.kea_only.test_kea_logging.test_v4_loggers_all_different_levels_different_file: kea #592
53: run tests-ubuntu-20.04-v4-tests.dhcpv4.kea_only.test_kea_logging.test_v4_loggers_all_different_levels_different_file: kea #592
54: run tests-fedora-34-v4-tests.dhcpv4.options.test_v4_vendor_options.test_v4_options_vendor_encapsulated_options_space_siemens: kea #1518
55: run tests-fedora-34-v4-tests.dhcpv4.options.test_v4_vendor_options.test_v4_options_vendor_encapsulated_siemens_defined_in_class: kea #1518
56: run tests-fedora-34-v4-tests.dhcpv4.options.test_v4_vendor_options.test_v4_options_vivso_suboptions_siemens_defined_in_class: kea #1518
57: run tests-ubuntu-20.04-v4-tests.dhcpv4.options.test_v4_vendor_options.test_v4_options_vendor_encapsulated_options_space_siemens: kea #1518
58: run tests-ubuntu-20.04-v4-tests.dhcpv4.options.test_v4_vendor_options.test_v4_options_vendor_encapsulated_siemens_defined_in_class: kea #1518
59: run tests-ubuntu-20.04-v4-tests.dhcpv4.options.test_v4_vendor_options.test_v4_options_vivso_suboptions_siemens_defined_in_class: kea #1518
60: run tests-ubuntu-20.04-cb-tests.dhcpv4.kea_only.config_backend.test_cb_v4_server_tag.test_server_tag_global_option4[mysql]: kea #1600
61: run tests-ubuntu-20.04-cb-tests.dhcpv4.kea_only.config_backend.test_cb_v4_server_tag.test_server_tag_global_option4[postgresql]: kea #1600
62: run tests-ubuntu-20.04-cb-tests.dhcpv6.kea_only.config_backend.test_cb_v6_server_tag.test_server_tag_global_option[mysql]: kea #1600
63: run tests-ubuntu-20.04-cb-tests.dhcpv6.kea_only.config_backend.test_cb_v6_server_tag.test_server_tag_global_option[postgresql]: kea #1600
64: run tests-fedora-34-v4-tests.dhcpv4.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-memfile]: kea #1796
65: run tests-fedora-34-v4-tests.dhcpv4.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-mysql]: kea #1796
66: run tests-fedora-34-v4-tests.dhcpv4.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql]: kea #1796
67: run tests-fedora-34-v6-tests.dhcpv6.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-memfile]: kea #1796
68: run tests-fedora-34-v6-tests.dhcpv6.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-mysql]: kea #1796
69: run tests-fedora-34-v6-tests.dhcpv6.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql]: kea #1796
70: run tests-ubuntu-20.04-v4-tests.dhcpv4.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-memfile]: kea #1796
71: run tests-ubuntu-20.04-v4-tests.dhcpv4.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-mysql]: kea #1796
72: run tests-ubuntu-20.04-v4-tests.dhcpv4.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql]: kea #1796
73: run tests-ubuntu-20.04-v6-tests.dhcpv6.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-memfile]: kea #1796
74: run tests-ubuntu-20.04-v6-tests.dhcpv6.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-mysql]: kea #1796
75: run tests-ubuntu-20.04-v6-tests.dhcpv6.kea_only.test_lease_cache.test_lease_cache_disabled[parameter1-postgresql]: kea #1796
76: run tests-fedora-34-v4-tests.dhcpv4.kea_only.leases_cmds.test_leases_cmds.test_hook_v4_lease_cmds_lease_get_negative: kea #2060
77: run tests-ubuntu-20.04-v4-tests.dhcpv4.kea_only.leases_cmds.test_leases_cmds.test_hook_v4_lease_cmds_lease_get_negative: kea #2060
78: run tests-ubuntu-20.04-v4-tests.dhcpv4.kea_only.test_backend_migration.test_v4_lease_upload_duplicate[mysql]: kea #2293
79: run tests-ubuntu-20.04-v4-tests.dhcpv4.kea_only.test_backend_migration.test_v4_lease_upload_duplicate[postgresql]: kea #2293
80: run tests-fedora-34-v4-tests.dhcpv4.kea_only.test_backend_migration.test_v4_lease_upload_duplicate[mysql]: kea #2293
81: run tests-fedora-34-v4-tests.dhcpv4.kea_only.test_backend_migration.test_v4_lease_upload_duplicate[postgresql]: kea #2293