12022-11-01 Aaron Merey <amerey@redhat.com> 2 3 * debuginfod-client.c (path_escape): Add early return. 4 52022-10-31 Aaron Merey <amerey@redhat.com> 6 7 * Makefile.am (libdebuginfod_so_LDLIBS): Add libelf. 8 * debuginfod-client.c (debuginfod_find_section): New function. 9 (path_escape): New function. 10 (extract_section): New function. 11 (cache_find_section): New function. 12 (debuginfod_query_server): Add support for section queries. 13 * debuginfod-find.c (main): Add support for section queries. 14 * debuginfod.cxx (extract_section): New function. 15 (handle_buildid_f_match): Add section parameter. When non-empty, 16 try to create response from section contents. 17 (handle_buildid_r_match): Add section parameter. When non-empty, 18 try to create response from section contents. 19 (handle_buildid_match): Add section parameter. Pass to 20 handle_buildid_{f,r}_match. 21 (handle_buildid): Handle section name when artifacttype is set to 22 "section". Query upstream servers via debuginfod_find_section 23 when necessary. 24 (debuginfod.h.in): Add declaration for debuginfod_find_section. 25 (libdebuginfod.map): Add debuginfod_find_section. 26 272022-10-18 Daniel Thornburgh <dthorn@google.com> 28 29 * debuginfod-client.c (debuginfod_query_server): Add DEBUGINFOD_HEADERS_FILE 30 setting to supply outgoing HTTP headers. 31 322022-10-17 Frank Ch. Eigler <fche@redhat.com> 33 34 * debuginfod.cxx (main): Report libmicrohttpd version. 35 362022-09-28 Aaron Merey <amerey@redhat.com> 37 38 * debuginfod-client.c (debuginfod_query_server): Switch sign of some 39 error codes from positive to negative. 40 412022-09-08 Frank Ch. Eigler <fche@redhat.com> 42 43 * debuginfod-client.c (debuginfod_query_server): Clear 44 ->winning_headers along with ->url at the start of any new query. 45 462022-09-08 Martin Liska <mliska@suse.cz> 47 48 * debuginfod-client.c (debuginfod_get_headers): Add to DUMMY_LIBDEBUGINFOD. 49 502022-09-06 Frank Ch. Eigler <fche@redhat.com> 51 52 * debuginfod-client.c (header_callback): Don't copy \r in x-d headers. 53 Print all headers in verbose_fd mode. 54 * debuginfod-find.c (parse_opt): Set verbose_fd only at verbosity >= 2. 55 * debuginfod.cxx (handle_buildid): Clean up header forwarding 56 string processing. 57 * debuginfod.h.in: (debuginfod_get_headers): Tweak wording. 58 * libdebuginfod.map: Use ELFUTILS_0.188 for new function. 59 602022-07-15 Noah Sanci <nsanci@redhat.com> 61 62 * debuginfod-client.c (header_callback): Ignore headers without 63 X-DEBUGINFOD prefix. 64 (debuginfod_query_server): Removed verbose printing headers when 65 undesired. 66 (debuginfod_get_headers): Created. 67 * debuginfod-find.c (main): Verbose printing headers. 68 * debuginfod.cxx (handle_buildid): Add headers prefixed with 69 X-DEBUGINFOD from federated servers to this server's response 70 headers. 71 * debuginfod.h.in (debuginfod_get_headers): Created. 72 * libdebuginfod.map: New elfutils version added. 73 742022-09-02 Aaron Merey <amerey@redhat.com> 75 76 * debuginfod.cxx (parse_opt): If '-C' is given with no arg, do not 77 update connection_pool since it will be done at a later point. 78 (main): Use auto-sized connection_pool if '-C' isn't given with an 79 arg. Do not use MHD_USE_THREAD_PER_CONNECTION. 80 812022-08-17 Martin Liska <mliska@suse.cz> 82 83 * debuginfod.cxx (handle_buildid): Update HTTP statistics only if 84 it comes from HTTP request. 85 862022-08-17 Martin Liska <mliska@suse.cz> 87 88 * debuginfod.cxx: Print filename for "cannot open archive". 89 902022-08-15 Frank Ch. Eigler <fche@redhat.com> 91 92 PR29474 93 * debuginfod.cxx (handle_buildid_r_match): Don't trigger false-404's 94 for concurrently prefetched target files. 95 962022-08-02 Josef Cejka <jcejka@suse.de> 97 98 * debuginfod.cxx (groom): Don't evaluate regex unless needed. 99 1002022-07-29 Josef Cejka <jcejka@suse.de> 101 102 * debuginfod.cxx: Create db indexes for fast delete while grooming. 103 1042022-06-03 Michael Trapp <michael.trapp@sap.com> 105 106 * debuginfod.cxx (scan_source_info): New global. 107 (parse_opt, elf_classify): Use it. 108 1092022-05-09 Mark Wielaard <mark@klomp.org> 110 111 * debuginfod-client.c (debuginfod_clean_cache): Move utime call to 112 before fts traversal. 113 1142022-05-09 Mark Wielaard <mark@klomp.org> 115 116 * debuginfod-client.c (debuginfod_init_cache): Remove. 117 (debuginfod_query_server): Don't call debuginfod_init_cache, call 118 mkdir then debuginfod_clean_cache. 119 1202022-05-09 Mark Wielaard <mark@klomp.org> 121 122 * debuginfod-client.c (debuginfod_config_cache): Always open with 123 O_CREATE first, then use fstat, only write the cache_config_default_s 124 value if st_size == 0, otherwise read value from file. 125 1262022-05-09 Mark Wielaard <mark@klomp.org> 127 128 * debuginfod.cxx (conninfo): Always provide servname to getnameinfo. 129 1302022-05-09 Mark Wielaard <mark@klomp.org> 131 132 * debuginfod-client.c (debuginfod_query_server): Add 133 curl_easy_setopt_ck macro, use it for all curl_easy_setopt calls. 134 1352022-05-09 Mark Wielaard <mark@klomp.org> 136 137 * debuginfod-client.c (debuginfod_write_callback): Check result 138 of curl_easy_getinfo. 139 1402022-05-05 Mark Wielaard <mark@klomp.org> 141 142 * debuginfod.cxx (main): Define use_epoll. Set to MHD_USE_EPOLL 143 based on MHD_VERSION. Don't use MHD_USE_THREAD_PER_CONNECTION 144 when use_poll is set. 145 1462022-05-05 Mark Wielaard <mark@klomp.org> 147 148 * debuginfod.cxx (main): Define mhd_flags. Use mhd_flags for 149 MHD_start_daemon. Try again with MHD_USE_DUAL_STACK removed if 150 that fails. Update clog to say either IPV4 or IPV4 and IPV6. 151 stop either ithe d46 or d4 daemonr. 152 1532022-05-09 Noah Sanci <nsanci@redhat.com> 154 155 * debuginfod.cxx (main): Set nonzero defaults for fdcache. 156 1572022-05-04 Frank Ch. Eigler <fche@redhat.com> 158 Mark Wielaard <mark@klomp.org> 159 160 * debuginfod-client.c (debuginfod_query_server): Correct fd leak 161 for cache negative-hit unlink case. 162 (debuginfod_config_cache, debuginfod_init_cache): Correct 163 minor fd leaks. 164 * debuginfod-find.c (main): Ditto. 165 1662022-04-22 Mark Wielaard <mark@klomp.org> 167 168 * Makefile.am (libdebuginfod): Add -lpthread. 169 (libdebuginfod_so_LDLIBS): Likewise. 170 * debuginfod-client.c (init_control): New static pthread_once_t. 171 (libcurl_init): New static function. 172 (debuginfod_begin): Use ptrace_once to call libcurl_init. 173 (libdebuginfod_ctor): Removed. 174 (libdebuginfod_dtor): Likewise. 175 1762022-04-24 Mark Wielaard <mark@klomp.org> 177 178 * debuginfod.cxx (main): Add MHD_USE_ITC to MHD_start_daemon flags. 179 1802022-04-13 Aaron Merey <amerey@redhat.com> 181 182 * debuginfod-client.c (debuginfod_query_server): 183 Drop st_mode check. Add st_size > 0 check. 184 Save target_mtime before calling 185 debuginfod_config_cache. unlink target_cache_path 186 on EACCESS. Create target_cache_path with DEFFILEMODE. 187 1882022-04-03 Frank Ch. Eigler <fche@redhat.com> 189 190 * debuginfod.cxx (main): Use single dual-stack daemon setup, 191 rather than duplicate ipv4 and ipv6. 192 (conninfo, handle_buildid): Represent ipv4-mapped ipv6 addresses 193 in their native ipv4 form for logging and X-F-F: purposes. 194 * debuginfod-client.c (debuginfod_add_http_header): Tolerate 195 colons in http header values. 196 1972022-04-03 Frank Ch. Eigler <fche@redhat.com> 198 199 * debuginfod.cxx (main): Use MHD_USE_EPOLL for libmicrohttpd, to 200 encourage more round-robin dispatch of incoming connections. 201 2022021-12-09 Alexander Kanavin <alex@linutronix.de> 203 204 * debuginfod-client.c (cache_clean_default_interval_s): Change type to 205 long from time_t. 206 (cache_miss_default_s): Likewise. 207 (cache_default_max_unused_age_s): Likewise. 208 2092021-12-09 Mark Wielaard <mark@klomp.org> 210 211 * debuginfod.cxx (database_stats_report): Don't format clog 212 using 'right' and 'setw(20)'. 213 2142021-12-08 Frank Ch. Eigler <fche@redhat.com> 215 216 * debuginfod.cxx (connection_pool): New global. 217 (parse_opt): Parse & check -C option to set it. 218 (error_cb): New callback for libmicrohttpd error counting. 219 (main): Activate MHD_OPTION_THREAD_POOL_SIZE if appropriate. 220 Activate error_cb. 221 2222021-12-04 Mark Wielaard <mark@klomp.org> 223 224 * debuginfod.cxx (main): Call debuginfod_pool_groom before exit. 225 2262021-12-08 Mark Wielaard <mark@klomp.org> 227 228 * debuginfod.cxx (add_mhd_response_header): New function. 229 (reportable_exception::mhd_send_response): Call 230 MHD_add_response_header. 231 (add_mhd_last_modified): Likewise. 232 (handle_buildid_f_match): Likewise. 233 (handle_buildid_r_match): Likewise. 234 (handle_metrics): Likewise. And check MHD_Response was actually 235 created. 236 (handle_root): Likewise. 237 2382021-12-08 Frank Ch. Eigler <fche@redhat.com> 239 240 * debuginfod.cxx (intern): Call set_metrics() holding the fdcache mutex. 241 2422021-12-04 Mark Wielaard <mark@klomp.org> 243 244 * debuginfod-client.c (debuginfod_query_server): Free winning_headers. 245 Reset response_data_size when clearing response_data. 246 2472021-12-01 Mark Wielaard <mark@klomp.org> 248 249 * debuginfod-client.c (debuginfod_query_server): Free tmp_url on 250 realloc error. curl_free escaped_string on error. Fix error out 251 goto on curl_easy_init failure. Only cleanup data[i] handle and 252 response_data if it was initialized. 253 2542021-12-01 Mark Wielaard <mark@klomp.org> 255 256 * debuginfod-client.c (timestamp): Use gmtime_r instead of gmtime. 257 (add_mhd_last_modified): Likewise. 258 2592021-11-10 Érico N. Rolim <erico.erc@gmail.com> 260 261 * debuginfod.cxx: include "system.h" under 'extern "C"' block. 262 2632021-11-05 Frank Ch. Eigler <fche@redhat.com> 264 265 PR28430 266 * debuginfod.cxx (parse_opt): Add "--passive" flag. Complain 267 about inconsistent flags. 268 (main): In passive mode, suppress scan/groom/traverse threads and 269 other read-write database ops. 270 2712021-11-04 Frank Ch. Eigler <fche@redhat.com> 272 273 PR28514 274 * debuginfod.cxx (groom): Rework into separate decision/action 275 phases. Add new metrics to monitor progress. Limit indefinite 276 operation times to avoid starving rescan. 277 2782021-10-23 Frank Ch. Eigler <fche@redhat.com> 279 280 PR28240 281 * debuginfod-client.c (debuginfod_query_server): Correct 282 negative-hit cache check sequence for root user. 283 2842021-10-15 Mark Wielaard <mark@klomp.org> 285 286 * debuginfod-client.c (debuginfod_query_server): Set 287 CURLOPT_PROTOCOLS. 288 2892021-10-06 Di Chen <dichen@redhat.com> 290 291 PR28242 292 * debuginfod.cxx (inc_metrics, add_metrics): Add two-tag variants. 293 (handler_cb): Call it with artifacttype for http_responses_* metrics. 294 (handle_buildid): Sanitize artifacttype if necessary. 295 (dwarf_extract_source_path): Pass sanitizable string param. 296 2972021-09-17 Noah Sanci <nsanci@redhat.com> 298 299 * debuginfod-client.c (debuginfod_query_server): curl_multi_perform 300 now occurs before checking if response headers have arrived. 301 3022021-09-14 Frank Ch. Eigler <fche@redhat.com> 303 304 PRPR28339 305 * debuginfod.cxx (waitq::fronters): New field. 306 (waitq::wait_idle): Respect it. 307 (waitq::done_front): New function. 308 (thread_main_scanner): Call it to match wait_front(). 309 3102021-09-12 Mark Wielaard <mark@klomp.org> 311 312 * debuginfod.cxx (libarchive_fdcache::lookup): Add endl after 313 obatched(clog) line. 3142021-09-13 Noah Sanci <nsanci@redhat.com> 315 316 * debuginfod-client.c (debuginfod_query_server): Removed constant 317 operations from a loop. curl_free memory. 318 3192021-09-06 Dmitry V. Levin <ldv@altlinux.org> 320 321 * debuginfod-client.c (debuginfod_begin): Remove cast of calloc return 322 value. 323 3242021-08-28 Mark Wielaard <mjw@redhat.com> 325 326 * debuginfod.cxx (parse_opt): Turn the -d arg ":memory:" into 327 "file::memory:?cache=shared" for the db_path. 328 3292021-08-20 Di Chen <dichen@redhat.com> 330 331 * debuginfod.cxx (options): Add ARGP_KEY_FORWARDED_TTL_LIMIT. 332 (forwarded_ttl_limit): New static unsigned. 333 (parse_opt): Handle ARGP_KEY_FORWARDED_TTL_LIMIT. 334 (handle_buildid): Check forwarded_ttl_limit. 335 (main): Log forwarded ttl limit. 336 3372021-08-20 Saleem Abdulrasool <abdulras@google.com> 338 339 * debuginfod.cxx: Remove error.h include. 340 3412021-08-19 Frank Ch. Eigler <fche@redhat.com> 342 343 PR28249 344 * debuginfod.cxx (handler_cb): Fix after_you unique_set key 345 to the entire incoming URL. 346 3472021-08-02 Noah Sanci <nsanci@redhat.com> 348 349 PR27277 350 * debuginfod-client.c (struct debuginfod_client): New field 351 winning_headers. 352 (struct handle_data): New field response_data, response_data_size. 353 (header_callback): Store received headers in response_data. 354 (debuginfod_query_server): Activate CURLOPT_HEADERFUNCTION. 355 Save winning response_data. 356 (debuginfod_end): free client winning headers. 357 * debuginfod.cxx (handle_buildid_f_match): remove X-DEBUGINFOD-FILE 358 path. Add X-DEBUGINFOD-FILE and X-DEBUGINFOD-SIZE headers. 359 (handle_buildid_r_match): remove X-DEBUGINFOD-FILE path. Add 360 X-DEBUGINFOD-FILE, X-DEBUGINFOD-SIZE 361 headers, and X-ARCHIVE headers. 362 3632021-07-26 Noah Sanci <nsanci@redhat.com> 364 365 PR27982 366 * debuginfod-client.c (globals): added default_maxsize and 367 default_maxtime. 368 (debuginfod_query_server): Added DEBUGINFOD_MAXSIZE and 369 DEBUGINFOD_MAXTIME envvar processing. 370 * debuginfod.cxx (handler_cb): If the requested file exceeds 371 maxsize return code 406. 372 * debuginfod.h.in: Added DEBUGINFOD_MAXSIZE_ENV_VAR and 373 DEBUGINFOD_MAXTIME_ENV_VAR. 374 3752021-07-16 Noah Sanci <nsanci@redhat.com> 376 377 PR28034 378 * debuginfod-client.c (debuginfod_query_server): % escape filename 379 so the completed url is processed properly. 380 3812021-06-28 Noah Sanci <nsanci@redhat.com> 382 383 PR25978 384 * debuginfod.cxx (options): Added --fdcache-prefetch-fds/mbs options. 385 (set_metric): Added a condition for fdcache_mintmp to ensure no 386 negative percentages or percentages larger than 100% are given. 387 (globals): Added fdcache_prefetch_mbs/fdcache_prefetch_fds. 388 (set_metrics): Differentiate between lru and prefetch metrics. 389 (intern): Added prefetch functionality for nuking preexisting copies 390 and incrementing prefetch metrics. 391 (lookup): Search prefetch cache and increment associated metrics. Upon 392 finding in the prefetch cache move the element to the lru cache. 393 (limit): Arguments updated. Update size of prefetch cache. 394 (main): Log prefetch and cache fds/mbs 395 3962021-07-06 Alice Zhang <alizhang@redhat.com> 397 398 PR27531 399 * debuginfod-client.c (debuginfod_query_server): Retry failed queries 400 if error code is not ENOENT. 401 * debuginfod.h.in: Introduce DEBUGINFOD_RETRY_LIMIT_ENV_VAR. 402 4032021-07-01 Noah Sanci <nsanci@redhat.com> 404 405 PR27711 406 * debuginfod.cxx (options): Add --regex-groom, -r option. 407 (regex_groom): New static bool defaults to false. 408 (parse_opt): Handle 'r' option by setting regex_groom to true. 409 (groom): Introduce and use reg_include and reg_exclude. 410 4112021-07-09 Noah Sanci <nsanci@redhat.com> 412 413 PR27983 414 * debuginfod-client.c (debuginfod_query_server): As full-length 415 urls are generated with standardized formats, ignore duplicates. 416 Created out1 and changed out2 error gotos. Updated url creation print 417 statements. 418 (globals): Removed url_delim_char, as it was no longer used. 419 4202021-06-18 Mark Wielaard <mark@klomp.org> 421 422 * debuginfod-client.c (debuginfod_begin): Don't use client if 423 calloc call failed. 424 4252021-06-03 Frank Ch. Eigler <fche@redhat.com> 426 427 PR27863 428 * debuginfod.cxx (unique_set, unique_set_reserver): New classes. 429 (handler_cb): Use them to implement "after-you" queueing. 430 4312021-05-14 Frank Ch. Eigler <fche@redhat.com> 432 433 PR27859 434 * debuginfod-client.c (debuginfod_client): Retain only 435 long-lived multi handle from PR27701 work. 436 (debuginfo_begin,debuginfod_end): ctor/dtor for surviving field only. 437 (debuginfod_query_server): Rework to reuse multi handle only. 438 4392021-04-19 Martin Liska <mliska@suse.cz> 440 441 * debuginfod-client.c (debuginfod_query_server): Use startswith. 442 (debuginfod_add_http_header): Likewise. 443 * debuginfod.cxx: Likewise. 444 4452021-05-04 Alice Zhang <alizhang@redhat.com> 446 447 * debuginfod-client.c (cache_miss_default_s): New static time_t, 448 defaults to 600 (10 minutes). 449 (cache_miss_filename): New static char pointer. 450 (debuginfod_config_cache): New static function. 451 (debuginfod_clean_cache): Use debuginfod_config_cache for 452 interval_path and max_unused_path. 453 (debuginfod_query_server): Check whether target_cache_path exists 454 as negative cache file and create target_cache_path when the server 455 returns ENOENT. Check cache_miss_path fir cache miss time. 456 4572021-04-26 Frank Ch. Eigler <fche@redhat.com> 458 459 PR27571 460 * debuginfod-client.c (debuginfod_query_server): Chmod 0400 files 461 delivered into the cache to prevent accidental modification. 462 4632021-04-26 Frank Ch. Eigler <fche@redhat.com> 464 465 PR26125 466 * debuginfod-client.c (debuginfod_clean_cache): For directory 467 rmdir, check mtime first. 468 (debuginfod_query_server): Try mkdir / mkstemp up to twice, 469 in case of race. 470 4712021-04-23 Frank Ch. Eigler <fche@redhat.com> 472 473 PR27701 474 * debuginfod-client.c (struct debuginfod_client): Add long-lived 475 CURL easy and multi handles. 476 (debuginfo_begin,debuginfod_end): ctor/dtor for these. 477 (debuginfod_query_server): Rework to reuse easy & multi handles. 478 (*_envvar): Just use the DEBUGINFOD_*_ENV_VAR directly instead. 479 480 * debuginfod.cxx (dc_pool): New pile of reusable debuginfod_client 481 objects for upstream federation connections. 482 (debuginfod_pool_{begin,end,groom}): New functions. 483 (handle_buildid): Use them. 484 (handler_cb): Fix keep-alive given libmicrohttpd convention of multiple 485 callbacks. 486 4872021-04-15 Frank Ch. Eigler <fche@redhat.com> 488 489 * debuginfod.cxx (groom): Only update database stats once. 490 4912021-04-15 Frank Ch. Eigler <fche@redhat.com> 492 493 * debuginfod.cxx (elf_classify): Recognize symtab-only stripped files 494 like fedora's libicudata as debuginfo files. 495 4962021-03-30 Frank Ch. Eigler <fche@redhat.com> 497 498 * debuginfod.cxx (main): Set child thread names. 499 5002021-03-07 Timm Bäder <tbaeder@redhat.com> 501 502 * debuginfod-client.c (debuginfod_query_server): Tweak 503 double/long clamping arithmetic to avoid UB and warnings. 504 5052021-02-25 Frank Ch. Eigler <fche@redhat.com> 506 507 * debuginfod.cxx (handler_cb): Filter webapi for bad 508 artifacttype keywords early for metric hygiene. 509 5102021-02-14 Frank Ch. Eigler <fche@redhat.com> 511 512 * debuginfod.cxx (main -U): Use bsdtar unconditionally. Also map 513 the debian-compatible .ipk (openembedded distro family) to same. 514 5152021-02-04 Frank Ch. Eigler <fche@redhat.com> 516 517 PR27092 low-memory handling 518 * debuginfod.cxx (fdcache_mintmp): New parameter, with cmd-line option. 519 (parse_opt): Parse it. 520 (main): Default it. 521 (statfs_free_enough_p): New function. 522 (libarchive_fdcache::*): Call it to trigger emergency fdcache flush. 523 (thread_main_scanner): Call it to report filesystem fullness metrics. 524 (groom): Ditto. 525 (set/add_metric): Take double rather than int64_t values. 526 (archive_exception): Propagate suberror to metric label. 527 (main): Detect pthread creation fatal errors properly. 528 5292021-02-02 Frank Ch. Eigler <fche@redhat.com> 530 531 PR27323 532 * debuginfod.cxx (dbq): New read-only database connection for queries 533 only. 534 (signal_handler): Interrupt it. 535 (main): Open / close it. 536 (handle_buildid): Use it for webapi queries only. 537 (database_stats_report): Make more interruptible. Report sqlite3 538 operation times to the prometheus metrics. 539 (groom): Make more interruptible. 540 (thread_main_fts_source_paths, thread_main_groom): Ensure 541 state/progress metrics are fresh even in case of exceptions. 542 5432020-12-20 Dmitry V. Levin <ldv@altlinux.org> 544 545 * .gitignore: New file. 546 5472020-12-12 Dmitry V. Levin <ldv@altlinux.org> 548 549 * debuginfod-client.c (debuginfod_query_server): Fix spelling typos in 550 comments. 551 * debuginfod.cxx: Likewise. 552 (parse_opt): Fix spelling typos in error diagnostics. 553 5542020-12-08 Dmitry V. Levin <ldv@altlinux.org> 555 556 * Makefile.am [LIBDEBUGINFOD]: Create libdebuginfod.so.1 first, turn 557 libdebuginfod.so into symlink. 558 5592020-11-30 Dmitry V. Levin <ldv@altlinux.org> 560 561 * Makefile.am (libdebuginfod.so): Replace $@.$(VERSION) with 562 $(LIBDEBUGINFOD_SONAME). 563 (install, uninstall, MOSTLYCLEANFILES): Replace 564 libdebuginfod.so.$(VERSION) with $(LIBDEBUGINFOD_SONAME). 565 (VERSION): Remove. 566 * debuginfod.h: Rename to ... 567 * debuginfod.h.in ... this. 568 (DEBUGINFOD_SONAME): New macro. 569 5702020-11-30 Dmitry V. Levin <ldv@altlinux.org> 571 572 * Makefile.am (libdebuginfod.so$(EXEEXT)): Drop $(EXEEXT) suffix. 573 5742020-11-25 Frank Ch. Eigler <fche@redhat.com> 575 576 * debuginfod.cxx (step_ok_done): Correct typo in prom metric label. 577 5782020-11-25 Frank Ch. Eigler <fche@redhat.com> 579 580 * debuginfod.cxx (tmp_ms_metric): Switch from gettimeofday to 581 clock_gettime(CLOCK_MONOTONIC) for time-interval measurements. 582 (handler_cb, scan_source_paths, groom): Ditto. 583 5842020-11-23 Frank Ch. Eigler <fche@redhat.com> 585 586 * debuginfod.cxx (tmp_ms_metric): New class for RAII timing metrics. 587 (sqlite_ps::reset, step*): Call it to track sqlite3 performance. 588 (sqlite_exception ctor): Increment sqlite3 error_count. 589 5902020-11-23 Mark Wielaard <mark@klomp.org> 591 592 * debuginfod-client.c (debuginfod_query_server): Initialize 593 struct handle_data errbuf to the empty string. 594 5952020-11-11 Mark Wielaard <mark@klomp.org> 596 597 * debuginfod-client.c (debuginfod_set_verbose_fd): New function. 598 (struct debuginfod_client): Add verbose_fd. 599 (struct handle_data): Add errbuf. 600 (debuginfod_query_server): Produce verbose output when 601 debuginfod_client verbose_fd is set. Only clear old data and set 602 default_headers when any work is done. Always goto out when setting 603 rc to an error value. Use CURLOPT_ERRORBUFFER to get more error 604 output when verbose output is requested. 605 * debuginfod.h (DEBUGINFOD_VERBOSE_ENV_VAR): New. 606 (debuginfod_set_verbose_fd): Added. 607 * debuginfod-find.c (parse_opt): Set debuginfod_set_verbose_fd on -v. 608 * bdebuginfod.map (ELFUTILS_0.183): New section, add 609 debuginfod_set_verbose_fd. 610 6112020-11-21 Mark Wielaard <mark@klomp.org> 612 613 * debuginfod.cxx (handle_root): New function. 614 (handler_cb): Handle "/" and report url1 in webapi error. 615 6162020-11-11 Mark Wielaard <mark@klomp.org> 617 618 * debuginfod-find.c (progressfn): Use clock_gettime to print Progress 619 at most 5 times a second. 620 6212020-11-19 Frank Ch. Eigler <fche@redhat.com> 622 623 * debuginfod.cxx (tmp_inc_metric): New class. 624 (handler_cb): Use it to track webapi operations. 625 6262020-11-01 Érico N. Rolim <erico.erc@gmail.com> 627 628 * debuginfod-client.c (debuginfod_init_cache): Use ACCESSPERMS for 629 mkdir, DEFFILEMODE for open with O_CREAT. 630 6312020-11-01 Érico N. Rolim <erico.erc@gmail.com> 632 633 * debuginfod.cxx: include libintl.h. 634 6352020-11-01 Érico N. Rolim <erico.erc@gmail.com> 636 637 * Makefile.am (debuginfod_LDADD): Add argp_LDADD and fts_LIBS. 638 (debuginfod_find_LDADD): Likewise. 639 (libdebuginfod_so_LDLIBS): Add fts_LIBS. 640 6412020-10-31 Frank Ch. Eigler <fche@redhat.com> 642 643 * debuginfod.cxx (scan_source_file, scan_archive_file): Add new scanned_bytes_total, 644 scanned_files_total metrics. 645 (archive_classify): Exit early if interrupted. 646 (scan_source_paths): Perform realpath/regex checks only on FTS_F files. 647 Tweak metrics. 648 6492020-10-30 Frank Ch. Eigler <fche@redhat.com> 650 651 PR26775 cont'd. 652 * debuginfod.cxx (thread_main_scanner): Ensure control doesn't 653 leave infinite loop until program exit, even if SIGUSR2. 654 (scan_source_paths): Have traverser clean scanq on 655 SIGUSR2. Emit additional traversed_total metrics. 656 (groom): Emit additional groomed_total metrics. 657 (thread_main_groom): Restore previous thread_work_total 658 metric. 659 6602020-10-29 Frank Ch. Eigler <fche@redhat.com> 661 662 PR26775 663 * debuginfod.cxx (forced_*_count): Make these global. 664 (runq::clear): New function. 665 (thread_main_scanner): Check for pending SIGUSR2; interrupt. 666 (scan_source_paths): Check for pending SIGUSR2; interrupt. 667 (groom): Report prometheus stats before groom also. Check for 668 pending SIGUSR1; interrupt. Increment thread_work_total for 669 each file scanned, not the entire cycle. 670 6712020-10-29 Frank Ch. Eigler <fche@redhat.com> 672 673 PR26810 674 * debuginfod.cxx (handle_buildid_*_match): Throw exceptions for 675 more lower level libc errors. 676 (handle_buildid_match): Catch & report exceptions but return 0 677 for continued iteration in the caller. 678 6792020-10-25 Mark Wielaard <mark@klomp.org> 680 681 * debuginfod-client.c (debuginfod_query_server): Translate 682 CURLE_PEER_FAILED_VERIFICATION to ECONNREFUSED. 683 6842020-10-20 Frank Ch. Eigler <fche@redhat.com> 685 686 PR26756: more prometheus metrics 687 * debuginfod.cxx (*_exception): Add counters for error occurrences. 688 (fdcache::*): Add counters for fdcache operations and status. 689 (fdcache::set_metric): New fn for overall stat counts. 690 (fdcache::limit): ... allow metric-less use from dtors. 691 6922020-10-20 Frank Ch. Eigler <fche@redhat.com> 693 694 * debuginfod.cxx (handle_buildid*): Add a parameter for detecting 695 internally-originated lookups for dwz resolution. 696 6972020-09-18 Frank Ch. Eigler <fche@redhat.com> 698 699 * debuginfod.cxx (scan_source_file, archive_classify): Store only 700 canonicalized file names in sdef & sref records in the database. 701 7022020-09-08 Mark Wielaard <mark@klomp.org> 703 704 * Makefile.am (BUILD_STATIC): Include libcurl_LIBS in libdebuginfod 705 when NOT DUMMY_LIBDEBUGINFOD. 706 7072020-09-16 Mark Wielaard <mark@klomp.org> 708 709 * debuginfod-find.c: Fix license block comment. 710 7112020-09-15 Mark Wielaard <mark@klomp.org> 712 713 * debuginfod-find.c (main): Use dwelf_elf_begin. 714 7152020-07-03 Alice Zhang <alizhang@redhat.com> 716 717 * debuginfod-client.c (debuginfod_query_server): Use strncasecmp 718 to compare effective_url. Try CURLINFO_SCHEME as fallback. 719 7202020-06-19 Mark Wielaard <mark@klomp.org> 721 722 * Makefile.am (bin_PROGRAMS): Guard with DEBUGINFOD and 723 LIBDEBUGINFOD. 724 (debuginfod_LDADD): Remove libcurl. 725 (libdebuginfod): When static and DUMMY_LIBDEBUGINFO remove libcurl. 726 (noinst_LIBRARIES): Guard with LIBDEBUGINFOD. 727 (AM_CPPFLAGS): Add -Wno-unused-parameter when DUMMY_LIBDEBUGINFOD. 728 (pkginclude_headers): Guard with LIBDEBUGINFOD 729 (libdebuginfod_so_LIBS): Likewise. 730 (+libdebuginfod_so_LDLIBS): Likewise. 731 (install): Likewise. 732 (uninstall): Likewise. 733 * debuginfod-client.c: Include dummy functions when 734 DUMMY_LIBDEBUGINFOD. 735 * debuginfod.cxx: Remove curl.h include. 736 7372020-06-16 Mark Wielaard <mark@klomp.org> 738 739 * debuginfod-client.c (debuginfod_query_server): Check malloc. 740 Move curl_multi_init call before handle_data malloc call. 741 7422020-06-16 Mark Wielaard <mark@klomp.org> 743 744 * debuginfod-client.c (debuginfod_query_server): Replace sizeof 745 build_id_bytes check with strlen build_id check. 746 7472020-06-16 Mark Wielaard <mark@klomp.org> 748 749 * debuginfod-client.c (debuginfod_query_server): Increase suffix 750 array and prepare having to escape 1 character with 2. 751 7522020-06-16 Mark Wielaard <mark@klomp.org> 753 754 * debuginfod-client.c (debuginfod_clean_cache): Handle failing 755 fopen (interval_path). 756 7572020-03-29 Mark Wielaard <mark@klomp.org> 758 759 * debuginfod-client.c (debuginfod_add_http_header): Check header 760 contains precisely one colon that isn't the first or last char. 761 7622020-03-29 Frank Ch. Eigler <fche@redhat.com> 763 764 * debuginfod-client.c (struct debuginfod_client): Add a flag field 765 for progressfn printing. 766 (default_progressfn): Set it if printing \rsomething. 767 (debuginfod_end): Terminate with \n if flag set, i.e., only if the 768 default_progressfn was actually called. 769 7702020-03-27 Mark Wielaard <mark@klomp.org> 771 772 * debuginfod.cxx (parse_opt): Check port is not zero. 773 7742020-03-28 Frank Ch. Eigler <fche@redhat.com> 775 776 * debuginfod.cxx (handle_buildid_r_match): During archive 777 extraction / fdcache prefetching, set the mtime of each 778 file in the cache. 779 7802020-03-27 Frank Ch. Eigler <fche@redhat.com> 781 782 * debuginfod-find.c (main): Extract buildid from /binary/ if 783 given instead of hex string. 784 * Makefile.am: Add elfutils library prereqs for debuginfod-find. 785 7862020-03-24 Frank Ch. Eigler <fche@redhat.com> 787 788 * debuginfod.h, libdebuginfod.map: New functions for _add_url_header. 789 * debuginfod-client.c (struct debuginfod_client): Add headers fields. 790 (debuginfod_add_http_header): New client api to add outgoing headers. 791 (add_default_headers): Renamed from add_extra_headers, skip if flag. 792 (debuginfod_query_server): Pass accumulated headers to libcurl. 793 (debuginfod_end): Clean accumulated headers. 794 (debuginfod_find_*): Add default headers at this point. 795 * debuginfod.cxx (handle_buildid): Add conn pointer. Use it to relay 796 incoming UA and XFF headers to federated upstream debuginfods. 797 7982020-03-26 Frank Ch. Eigler <fche@redhat.com> 799 800 * debuginfod.cxx (handler_cb): Export two families of metrics for 801 prometheus traffic analysis: response times and data amounts. 802 8032020-03-26 Frank Ch. Eigler <fche@redhat.com> 804 805 * debuginfod.cxx (parse_opt): For -U, prefer dpkg-deb 806 after all if access(3)-able, fallback to bsdtar. 807 8082020-03-25 Frank Ch. Eigler <fche@redhat.com> 809 810 * debuginfod.cxx (parse_opt): Associate a bsdtar subshell with 811 the .deb & .ddeb extensions, instead of dpkg-deb. 812 8132020-03-26 Frank Ch. Eigler <fche@redhat.com> 814 815 * debuginfod-client.c (debuginfod_query_server): Don't 816 set CURLOPT_PATH_AS_IS on old curl. Mostly harmless. 817 8182020-03-24 Frank Ch. Eigler <fche@redhat.com> 819 820 * debuginfod-client.c (debuginfod_query_server): Set 821 CURLOPT_PATH_AS_IS, to propagate file names verbatim. 822 * debuginfod.cxx (canon_pathname): Implement RFC3986 823 style pathname canonicalization. 824 (handle_buildid): Canonicalize incoming webapi source 825 paths, accept either one. 826 (scan_source_file, archive_classify): Store both 827 original and canonicalized dwarf-source file names. 828 8292020-03-24 Frank Ch. Eigler <fche@redhat.com> 830 831 * debuginfod.cxx (handle_buildid): In case of federated fallback 832 queries, handle errors analogously to local ENOENT/404. 833 (handle_metrics): Return a size-of-response value. 834 (handler_cb): Add code to time entire application-side processing 835 stage + response sizes + http codes, so as to emit a complete 836 httpd-flavoured log line for each webapi request. 837 8382020-03-24 Frank Ch. Eigler <fche@redhat.com> 839 840 * debuginfod-client.c (debuginfod_query_server): Print the 841 default_progressfn terminating \n message only if that progressfn 842 is actually set. 843 8442020-03-24 Frank Ch. Eigler <fche@redhat.com> 845 846 * debuginfod-find.c (main): Correct /source full-pathness check for 847 "debuginfod-find -v source deadbeef /pathname" case. 848 8492020-03-22 Frank Ch. Eigler <fche@redhat.com> 850 851 * debuginfod-client.c (struct debuginfod_client): Add url field. 852 (struct handle_data): Add client field as backpointer. 853 (debuginfod_write_callback): Compute & save URL. 854 (default_progressfn): Print front pieces of the URL. 855 (debuginfod_query_server): Clear URL and cleanup after progressfn. 856 * debuginfod-find.c (main): Print URL at transfer conclusion. 857 8582020-03-22 Frank Ch. Eigler <fche@redhat.com> 859 860 * debuginfod.h, libdebuginfod.map: New functions for _get/set_user(). 861 * debuginfod-client.c: Implement them. 862 * debuginfod-find.c: Include a token call just for testing them. 863 8642020-03-03 Aaron Merey <amerey@redhat.com> 865 866 * debuginfod-client.c (debuginfod_query_server): Update 867 cache_path even when new default path already exists. 868 8692020-02-27 Aaron Merey <amerey@redhat.com> 870 871 * debuginfod-client.c (xalloc_str): New macro. Call 872 asprintf with error checking. 873 (debuginfod_query_server): Use XDG_CACHE_HOME as a default 874 cache location if it is set. Replace snprintf with xalloc_str. 875 8762020-02-26 Konrad Kleine <kkleine@redhat.com> 877 878 * debuginfod-client.c (debuginfod_query_server): Handle curl's 879 response code correctly when DEBUGINFOD_URLS begin with file:// 880 8812020-02-25 Frank Ch. Eigler <fche@redhat.com> 882 883 * debuginfod.cxx (parse_opt): Treat -R as if -Z.rpm . 884 8852020-02-25 Frank Ch. Eigler <fche@redhat.com> 886 887 * debuginfod.cxx (fdcache_prefetch): New parameter. 888 (parse_opt): Parse it. 889 (main): Default it. 890 (fdcache::fd_size_mb): Change to double for accuracy. 891 (fdcache::probe): New function. 892 (fdcache::intern): New option to intern at end of LRU. 893 (fdcache::lookup): Clean fdcache. 894 (handle_buildid_r_match): Implement multi-stage archive 895 parsing, with optional prefetching of extracted contents 896 into the fdcache. 897 8982020-02-19 Aaron Merey <amerey@redhat.com> 899 900 * debuginfod-client.c (debuginfod_clean_cache): Restrict 901 cleanup to client-pattern files. 902 9032020-02-05 Frank Ch. Eigler <fche@redhat.com> 904 905 * debuginfod.cxx (argp options): Add -Z option. 906 (canonicalized_archive_entry_pathname): New function for 907 distro-agnostic file name matching/storage. 908 9092020-01-22 Frank Ch. Eigler <fche@redhat.com> 910 911 * debuginfod.cxx (dwarf_extract_source_paths): Don't print 912 "skipping hat" messages at verbosity <=3, too noisy. 913 9142020-01-19 Frank Ch. Eigler <fche@redhat.com> 915 916 * debuginfod.cxx (scanq): Rework to let groomer/fts threads 917 synchronize with an empty workqueue, and lock out workqueue 918 consumers. 919 (thread_groom): Adopt new scanq idle APIs to lock out scanners. 920 (thread_main_fts_source_paths): Adopt new scanq idler API to 921 avoid being restarted while scanners haven't even finished yet. 922 (thread_main_*): Increment thread_work_total metric only after 923 a work cycle is completed, not when it begins. 924 9252020-01-18 Frank Ch. Eigler <fche@redhat.com> 926 927 * debuginfod.cxx (thread_main_scanner): Handle empty source_paths[]. 928 9292020-01-11 Frank Ch. Eigler <fche@redhat.com> 930 931 * debuginfod.cxx (libarchive_fdcache): New class/facility to own a 932 cache of temporary files that were previously extracted from an 933 archive. If only it could store just unlinked fd's instead of 934 filenames. 935 (handle_buildid_r_match): Use it to answer dwz/altdebug and webapi 936 requests. 937 (groom): Clean it. 938 (main): Initialize the cache control parameters from heuristics. 939 Use a consistent tmpdir for these and tmp files elsewhere. 940 9412020-01-11 Frank Ch. Eigler <fche@redhat.com> 942 943 * debuginfod.cxx (conninfo): Print User-Agent and X-Forwarded-For 944 request headers, after mild safety-censorship (for easier machine 945 processing). 946 9472020-01-11 Frank Ch. Eigler <fche@redhat.com> 948 949 * debuginfod.cxx: Rework threading model. 950 (workq): New class for concurrent work-queue. 951 (semaphore): Removed class, now unused. 952 (scan_source_file_path): Rework into ... 953 (scan_source_file): New function. 954 (thread_main_scan_source_file_path): Nuke. 955 (scan_source_archive_path): Rework into ... 956 (scan_archive_file): New function. 957 (thread_main_scanner): New function for scanner threads. 958 (thread_main_fts_source_paths): New function for traversal thread. 959 (scan_source_paths): ... doing this. 960 (thread_groom): Tweak metrics for consistency. 961 (main): Start 1 traversal and N scanner threads if needed. 962 9632019-01-02 Mark Wielaard <mark@klomp.org> 964 965 * debuginfod.cxx (default_connect_timeout): Removed. 966 (default_transfer_timeout): Removed. 967 (default_timeout): New. Default to 90 seconds. 968 (debuginfod_query_server): Parse server_timeout_envvar as one number. 969 Set as CURLOPT_LOW_SPEED_TIME, with CURL_OPT_LOW_SPEED_LIMITE as 100K. 970 9712020-01-09 Frank Ch. Eigler <fche@redhat.com> 972 973 * debuginfod-client.c (add_extra_headers): New function, 974 based on mjw's draft. 975 (debuginfod_query_server): Call it. 976 9772019-12-22 Frank Ch. Eigler <fche@redhat.com> 978 979 * debuginfod.cxx (*_rpm_*): Rename to *_archive_* throughout. 980 (scan_archives): New read-mostly global to identify archive 981 file extensions and corresponding extractor commands. 982 (parse_opt): Handle new -U flag. 983 9842019-12-19 Frank Ch. Eigler <fche@redhat.com> 985 986 * debuginfod-client.c (default_progressfn): New function. 987 (debuginfod_begin): Use it if $DEBUGINFOD_PROGRESS set. 988 (server_timeout): Bump to 30 seconds. 989 (debuginfod_query_server): Call progressfn -after- rather than 990 before curl ops, to make it likely that a successful transfer 991 results in final a=b call. Tweak cleanup sequence. 992 * debuginfod.h: Document $DEBUGINFOD_PROGRESS name. 993 9942019-12-09 Mark Wielaard <mark@klomp.org> 995 996 * debuginfod-client.c (debuginfod_query_server): Check 997 server_urls_envvar early. 998 9992019-12-03 Mark Wielaard <mark@klomp.org> 1000 1001 * debuginfod-client.c (debuginfod_query_server): Use separate 1002 local variables for CURLcode curl_res and CURLMcode curlm_res. 1003 10042019-11-26 Mark Wielaard <mark@klomp.org> 1005 1006 * Makefile.am (BUILD_STATIC): Add needed libraries for libdw and 1007 libdebuginfod. 1008 10092019-11-25 Frank Ch. Eigler <fche@redhat.com> 1010 1011 * debuginfod.cxx (groom): Add a sqlite3_db_release_memory() 1012 at the end of periodic grooming to try to shrink the process. 1013 10142019-11-24 Mark Wielaard <mark@klomp.org> 1015 1016 * debuginfod.cxx (test_webapi_sleep): Removed. 1017 (handler_cb): Don't check test_webapi_sleep and sleep. 1018 (main): Don't set test_webapi_sleep. 1019 10202019-11-24 Mark Wielaard <mark@klomp.org> 1021 1022 * debuginfod.cxx (add_metric): New function. 1023 (scan_source_file_path): Record metrics for 1024 found_executable_total, found_debuginfo_total and 1025 found_sourcerefs_total. 1026 (scan_source_rpm_path): Likewise. 1027 10282019-11-07 Frank Ch. Eigler <fche@redhat.com> 1029 1030 * debuginfod.cxx: Add /metrics endpoint. Add numerous 1031 calls to new functions inc_metric/set_metric to populate 1032 threadsafe map containing stats. Add http content-type 1033 response headers throughout. 1034 (thread_main_*): Simplify counter/timer flow. 1035 (main): Reorder web service shutdown to leave http running 1036 as long as possible. 1037 * debuginfod.8: Document it, add security caution. 1038 10392019-11-06 Frank Ch. Eigler <fche@redhat.com> 1040 1041 * debuginfod.cxx: Add new -L (symlink-following) mode. 1042 * debuginfod.8: Document it. 1043 10442019-11-04 Frank Ch. Eigler <fche@redhat.com> 1045 1046 * debuginfo-client.c (debuginfod_set_progressfn): New function 1047 for progress/interrupt callback. 1048 (debuginfod_clean_cache, debuginfod_query_server): Call it. 1049 * debuginfo.h: Declare it. 1050 * debuginfod_set_progressfn.3, *_find_debuginfo.3: Document it. 1051 * Makefile.am: Install it. 1052 * libdebuginfod.map: Export it all under ELFUTILS_0.178 symversion. 1053 1054 * debuginfod-find.c: Add -v option to activate progress cb. 1055 * debuginfod-find.1: Document it. 1056 * debuginfod.cxx: Add $DEBUGINFOD_TEST_WEBAPI_SLEEP env var 1057 to insert sleep in webapi callbacks, to help manual testing. 1058 10592019-10-28 Frank Ch. Eigler <fche@redhat.com> 1060 1061 * debuginfod.cxx: New file: debuginfod server. 1062 * debuginfod.8: New file: man page. 1063 * Makefile.am: Build it. 1064 10652019-10-28 Aaron Merey <amerey@redhat.com> 1066 1067 * debuginfod-client.c: New file: debuginfod client library. 1068 * debuginfod.h: New file: header for same. 1069 * libdebuginfod.map: New file: govern its solib exports. 1070 * debuginfod-find.c: New file: command line frontend. 1071 * debuginfod-find.1, debuginfod_find_source.3, 1072 debuginfod_find_executable.3, debuginfod_find_debuginfo.3: 1073 New man pages. 1074