• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
12022-11-01  Aaron Merey  <amerey@redhat.com>
2
3	* debuginfod-client.c (path_escape): Add early return.
4
52022-10-31  Aaron Merey  <amerey@redhat.com>
6
7	* Makefile.am (libdebuginfod_so_LDLIBS): Add libelf.
8	* debuginfod-client.c (debuginfod_find_section): New function.
9	(path_escape): New function.
10	(extract_section): New function.
11	(cache_find_section): New function.
12	(debuginfod_query_server): Add support for section queries.
13	* debuginfod-find.c (main): Add support for section queries.
14	* debuginfod.cxx (extract_section): New function.
15	(handle_buildid_f_match): Add section parameter.  When non-empty,
16	try to create response from section contents.
17	(handle_buildid_r_match): Add section parameter.  When non-empty,
18	try to create response from section contents.
19	(handle_buildid_match): Add section parameter. Pass to
20	handle_buildid_{f,r}_match.
21	(handle_buildid): Handle section name when artifacttype is set to
22	"section".  Query upstream servers via debuginfod_find_section
23	when necessary.
24	(debuginfod.h.in): Add declaration for debuginfod_find_section.
25	(libdebuginfod.map): Add debuginfod_find_section.
26
272022-10-18  Daniel Thornburgh <dthorn@google.com>
28
29  * debuginfod-client.c (debuginfod_query_server): Add DEBUGINFOD_HEADERS_FILE
30  setting to supply outgoing HTTP headers.
31
322022-10-17  Frank Ch. Eigler  <fche@redhat.com>
33
34	* debuginfod.cxx (main): Report libmicrohttpd version.
35
362022-09-28  Aaron Merey  <amerey@redhat.com>
37
38	* debuginfod-client.c (debuginfod_query_server): Switch sign of some
39	error codes from positive to negative.
40
412022-09-08  Frank Ch. Eigler  <fche@redhat.com>
42
43	* debuginfod-client.c (debuginfod_query_server): Clear
44	->winning_headers along with ->url at the start of any new query.
45
462022-09-08  Martin Liska  <mliska@suse.cz>
47
48	* debuginfod-client.c (debuginfod_get_headers): Add to DUMMY_LIBDEBUGINFOD.
49
502022-09-06  Frank Ch. Eigler  <fche@redhat.com>
51
52	* debuginfod-client.c (header_callback): Don't copy \r in x-d headers.
53	Print all headers in verbose_fd mode.
54	* debuginfod-find.c (parse_opt): Set verbose_fd only at verbosity >= 2.
55	* debuginfod.cxx (handle_buildid): Clean up header forwarding
56	string processing.
57	* debuginfod.h.in: (debuginfod_get_headers): Tweak wording.
58	* libdebuginfod.map: Use ELFUTILS_0.188 for new function.
59
602022-07-15  Noah Sanci  <nsanci@redhat.com>
61
62	* debuginfod-client.c (header_callback): Ignore headers without
63	X-DEBUGINFOD prefix.
64	(debuginfod_query_server): Removed verbose printing headers when
65	undesired.
66	(debuginfod_get_headers): Created.
67	* debuginfod-find.c (main): Verbose printing headers.
68	* debuginfod.cxx (handle_buildid): Add headers prefixed with
69	X-DEBUGINFOD from federated servers to this server's response
70	headers.
71	* debuginfod.h.in (debuginfod_get_headers): Created.
72	* libdebuginfod.map: New elfutils version added.
73
742022-09-02  Aaron Merey  <amerey@redhat.com>
75
76	* debuginfod.cxx (parse_opt): If '-C' is given with no arg, do not
77	update connection_pool since it will be done at a later point.
78	(main): Use auto-sized connection_pool if '-C' isn't given with an
79	arg. Do not use MHD_USE_THREAD_PER_CONNECTION.
80
812022-08-17  Martin Liska  <mliska@suse.cz>
82
83	* debuginfod.cxx (handle_buildid): Update HTTP statistics only if
84	it comes from HTTP request.
85
862022-08-17  Martin Liska  <mliska@suse.cz>
87
88	* debuginfod.cxx: Print filename for "cannot open archive".
89
902022-08-15  Frank Ch. Eigler <fche@redhat.com>
91
92	PR29474
93	* debuginfod.cxx (handle_buildid_r_match): Don't trigger false-404's
94	for concurrently prefetched target files.
95
962022-08-02  Josef Cejka <jcejka@suse.de>
97
98	* debuginfod.cxx (groom): Don't evaluate regex unless needed.
99
1002022-07-29  Josef Cejka <jcejka@suse.de>
101
102	* debuginfod.cxx: Create db indexes for fast delete while grooming.
103
1042022-06-03  Michael Trapp <michael.trapp@sap.com>
105
106	* debuginfod.cxx (scan_source_info): New global.
107	(parse_opt, elf_classify): Use it.
108
1092022-05-09  Mark Wielaard  <mark@klomp.org>
110
111	* debuginfod-client.c (debuginfod_clean_cache): Move utime call to
112	before fts traversal.
113
1142022-05-09  Mark Wielaard  <mark@klomp.org>
115
116	* debuginfod-client.c (debuginfod_init_cache): Remove.
117	(debuginfod_query_server): Don't call debuginfod_init_cache, call
118	mkdir then debuginfod_clean_cache.
119
1202022-05-09  Mark Wielaard  <mark@klomp.org>
121
122	* debuginfod-client.c (debuginfod_config_cache): Always open with
123	O_CREATE first, then use fstat, only write the cache_config_default_s
124	value if st_size == 0, otherwise read value from file.
125
1262022-05-09  Mark Wielaard  <mark@klomp.org>
127
128	* debuginfod.cxx (conninfo): Always provide servname to getnameinfo.
129
1302022-05-09  Mark Wielaard  <mark@klomp.org>
131
132	* debuginfod-client.c (debuginfod_query_server): Add
133	curl_easy_setopt_ck macro, use it for all curl_easy_setopt calls.
134
1352022-05-09  Mark Wielaard  <mark@klomp.org>
136
137	* debuginfod-client.c (debuginfod_write_callback): Check result
138	of curl_easy_getinfo.
139
1402022-05-05  Mark Wielaard  <mark@klomp.org>
141
142	* debuginfod.cxx (main): Define use_epoll. Set to MHD_USE_EPOLL
143	based on MHD_VERSION. Don't use MHD_USE_THREAD_PER_CONNECTION
144	when use_poll is set.
145
1462022-05-05  Mark Wielaard  <mark@klomp.org>
147
148	* debuginfod.cxx (main): Define mhd_flags. Use mhd_flags for
149	MHD_start_daemon. Try again with MHD_USE_DUAL_STACK removed if
150	that fails. Update clog to say either IPV4 or IPV4 and IPV6.
151	stop either ithe d46 or d4 daemonr.
152
1532022-05-09  Noah Sanci  <nsanci@redhat.com>
154
155	* debuginfod.cxx (main): Set nonzero defaults for fdcache.
156
1572022-05-04  Frank Ch. Eigler <fche@redhat.com>
158	    Mark Wielaard  <mark@klomp.org>
159
160	* debuginfod-client.c (debuginfod_query_server): Correct fd leak
161	for cache negative-hit unlink case.
162	(debuginfod_config_cache, debuginfod_init_cache): Correct
163	minor fd leaks.
164	* debuginfod-find.c  (main): Ditto.
165
1662022-04-22  Mark Wielaard  <mark@klomp.org>
167
168	* Makefile.am (libdebuginfod): Add -lpthread.
169	(libdebuginfod_so_LDLIBS): Likewise.
170	* debuginfod-client.c (init_control): New static pthread_once_t.
171	(libcurl_init): New static function.
172	(debuginfod_begin): Use ptrace_once to call libcurl_init.
173	(libdebuginfod_ctor): Removed.
174	(libdebuginfod_dtor): Likewise.
175
1762022-04-24  Mark Wielaard  <mark@klomp.org>
177
178	* debuginfod.cxx (main): Add MHD_USE_ITC to MHD_start_daemon flags.
179
1802022-04-13  Aaron Merey  <amerey@redhat.com>
181
182	* debuginfod-client.c (debuginfod_query_server):
183	Drop st_mode check. Add st_size > 0 check.
184	Save target_mtime before calling
185	debuginfod_config_cache. unlink target_cache_path
186	on EACCESS. Create target_cache_path with DEFFILEMODE.
187
1882022-04-03  Frank Ch. Eigler <fche@redhat.com>
189
190	* debuginfod.cxx (main): Use single dual-stack daemon setup,
191	rather than duplicate ipv4 and ipv6.
192	(conninfo, handle_buildid): Represent ipv4-mapped ipv6 addresses
193	in their native ipv4 form for logging and X-F-F: purposes.
194	* debuginfod-client.c (debuginfod_add_http_header): Tolerate
195	colons in http header values.
196
1972022-04-03  Frank Ch. Eigler <fche@redhat.com>
198
199	* debuginfod.cxx (main): Use MHD_USE_EPOLL for libmicrohttpd, to
200	encourage more round-robin dispatch of incoming connections.
201
2022021-12-09  Alexander Kanavin <alex@linutronix.de>
203
204	* debuginfod-client.c (cache_clean_default_interval_s): Change type to
205	long from time_t.
206	(cache_miss_default_s): Likewise.
207	(cache_default_max_unused_age_s): Likewise.
208
2092021-12-09  Mark Wielaard  <mark@klomp.org>
210
211	* debuginfod.cxx (database_stats_report): Don't format clog
212	using 'right' and 'setw(20)'.
213
2142021-12-08  Frank Ch. Eigler  <fche@redhat.com>
215
216	* debuginfod.cxx (connection_pool): New global.
217	(parse_opt): Parse & check -C option to set it.
218	(error_cb): New callback for libmicrohttpd error counting.
219	(main): Activate MHD_OPTION_THREAD_POOL_SIZE if appropriate.
220	Activate error_cb.
221
2222021-12-04  Mark Wielaard  <mark@klomp.org>
223
224	* debuginfod.cxx (main): Call debuginfod_pool_groom before exit.
225
2262021-12-08  Mark Wielaard  <mark@klomp.org>
227
228	* debuginfod.cxx (add_mhd_response_header): New function.
229	(reportable_exception::mhd_send_response): Call
230	MHD_add_response_header.
231	(add_mhd_last_modified): Likewise.
232	(handle_buildid_f_match): Likewise.
233	(handle_buildid_r_match): Likewise.
234	(handle_metrics): Likewise. And check MHD_Response was actually
235	created.
236	(handle_root): Likewise.
237
2382021-12-08  Frank Ch. Eigler  <fche@redhat.com>
239
240	* debuginfod.cxx (intern): Call set_metrics() holding the fdcache mutex.
241
2422021-12-04  Mark Wielaard  <mark@klomp.org>
243
244	* debuginfod-client.c (debuginfod_query_server): Free winning_headers.
245	Reset response_data_size when clearing response_data.
246
2472021-12-01  Mark Wielaard  <mark@klomp.org>
248
249	* debuginfod-client.c (debuginfod_query_server): Free tmp_url on
250	realloc error. curl_free escaped_string on error. Fix error out
251	goto on curl_easy_init failure. Only cleanup data[i] handle and
252	response_data if it was initialized.
253
2542021-12-01  Mark Wielaard  <mark@klomp.org>
255
256	* debuginfod-client.c (timestamp): Use gmtime_r instead of gmtime.
257	(add_mhd_last_modified): Likewise.
258
2592021-11-10  Érico N. Rolim  <erico.erc@gmail.com>
260
261	* debuginfod.cxx: include "system.h" under 'extern "C"' block.
262
2632021-11-05  Frank Ch. Eigler  <fche@redhat.com>
264
265	PR28430
266	* debuginfod.cxx (parse_opt): Add "--passive" flag.  Complain
267	about inconsistent flags.
268	(main): In passive mode, suppress scan/groom/traverse threads and
269	other read-write database ops.
270
2712021-11-04  Frank Ch. Eigler  <fche@redhat.com>
272
273	PR28514
274	* debuginfod.cxx (groom): Rework into separate decision/action
275	phases.  Add new metrics to monitor progress.  Limit indefinite
276	operation times to avoid starving rescan.
277
2782021-10-23  Frank Ch. Eigler  <fche@redhat.com>
279
280	PR28240
281	* debuginfod-client.c (debuginfod_query_server): Correct
282	negative-hit cache check sequence for root user.
283
2842021-10-15  Mark Wielaard  <mark@klomp.org>
285
286	* debuginfod-client.c (debuginfod_query_server): Set
287	CURLOPT_PROTOCOLS.
288
2892021-10-06  Di Chen <dichen@redhat.com>
290
291	PR28242
292	* debuginfod.cxx (inc_metrics, add_metrics): Add two-tag variants.
293	(handler_cb): Call it with artifacttype for http_responses_* metrics.
294	(handle_buildid): Sanitize artifacttype if necessary.
295	(dwarf_extract_source_path): Pass sanitizable string param.
296
2972021-09-17  Noah Sanci  <nsanci@redhat.com>
298
299	* debuginfod-client.c (debuginfod_query_server): curl_multi_perform
300	now occurs before checking if response headers have arrived.
301
3022021-09-14  Frank Ch. Eigler <fche@redhat.com>
303
304	PRPR28339
305	* debuginfod.cxx (waitq::fronters): New field.
306	(waitq::wait_idle): Respect it.
307	(waitq::done_front): New function.
308	(thread_main_scanner): Call it to match wait_front().
309
3102021-09-12  Mark Wielaard  <mark@klomp.org>
311
312	* debuginfod.cxx (libarchive_fdcache::lookup): Add endl after
313	obatched(clog) line.
3142021-09-13  Noah Sanci  <nsanci@redhat.com>
315
316	* debuginfod-client.c (debuginfod_query_server): Removed constant
317	operations from a loop. curl_free memory.
318
3192021-09-06  Dmitry V. Levin  <ldv@altlinux.org>
320
321	* debuginfod-client.c (debuginfod_begin): Remove cast of calloc return
322	value.
323
3242021-08-28  Mark Wielaard  <mjw@redhat.com>
325
326	* debuginfod.cxx (parse_opt): Turn the -d arg ":memory:" into
327	"file::memory:?cache=shared" for the db_path.
328
3292021-08-20  Di Chen  <dichen@redhat.com>
330
331	* debuginfod.cxx (options): Add ARGP_KEY_FORWARDED_TTL_LIMIT.
332	(forwarded_ttl_limit): New static unsigned.
333	(parse_opt): Handle ARGP_KEY_FORWARDED_TTL_LIMIT.
334	(handle_buildid): Check forwarded_ttl_limit.
335	(main): Log forwarded ttl limit.
336
3372021-08-20  Saleem Abdulrasool  <abdulras@google.com>
338
339	* debuginfod.cxx: Remove error.h include.
340
3412021-08-19  Frank Ch. Eigler  <fche@redhat.com>
342
343	PR28249
344	* debuginfod.cxx (handler_cb): Fix after_you unique_set key
345	to the entire incoming URL.
346
3472021-08-02  Noah Sanci  <nsanci@redhat.com>
348
349	PR27277
350	* debuginfod-client.c (struct debuginfod_client): New field
351	winning_headers.
352	(struct handle_data): New field response_data, response_data_size.
353	(header_callback): Store received headers in response_data.
354	(debuginfod_query_server): Activate CURLOPT_HEADERFUNCTION.
355	Save winning response_data.
356	(debuginfod_end): free client winning headers.
357	* debuginfod.cxx (handle_buildid_f_match): remove X-DEBUGINFOD-FILE
358	path. Add X-DEBUGINFOD-FILE and X-DEBUGINFOD-SIZE headers.
359	(handle_buildid_r_match): remove X-DEBUGINFOD-FILE path. Add
360	X-DEBUGINFOD-FILE, X-DEBUGINFOD-SIZE
361	headers, and X-ARCHIVE headers.
362
3632021-07-26  Noah Sanci  <nsanci@redhat.com>
364
365	PR27982
366	* debuginfod-client.c (globals): added default_maxsize and
367	default_maxtime.
368	(debuginfod_query_server): Added DEBUGINFOD_MAXSIZE and
369	DEBUGINFOD_MAXTIME envvar processing.
370	* debuginfod.cxx (handler_cb): If the requested file exceeds
371	maxsize return code 406.
372	* debuginfod.h.in: Added DEBUGINFOD_MAXSIZE_ENV_VAR and
373	DEBUGINFOD_MAXTIME_ENV_VAR.
374
3752021-07-16  Noah Sanci  <nsanci@redhat.com>
376
377	PR28034
378	* debuginfod-client.c (debuginfod_query_server): % escape filename
379	so the completed url is processed properly.
380
3812021-06-28 Noah Sanci <nsanci@redhat.com>
382
383	PR25978
384	* debuginfod.cxx (options): Added --fdcache-prefetch-fds/mbs options.
385	(set_metric): Added a condition for fdcache_mintmp to ensure no
386	negative percentages or percentages larger than 100% are given.
387	(globals): Added fdcache_prefetch_mbs/fdcache_prefetch_fds.
388	(set_metrics): Differentiate between lru and prefetch metrics.
389	(intern): Added prefetch functionality for nuking preexisting copies
390	and incrementing prefetch metrics.
391	(lookup): Search prefetch cache and increment associated metrics. Upon
392	finding in the prefetch cache move the element to the lru cache.
393	(limit): Arguments updated. Update size of prefetch cache.
394	(main): Log prefetch and cache fds/mbs
395
3962021-07-06  Alice Zhang  <alizhang@redhat.com>
397
398	PR27531
399	* debuginfod-client.c (debuginfod_query_server): Retry failed queries
400	if error code is not ENOENT.
401	* debuginfod.h.in: Introduce DEBUGINFOD_RETRY_LIMIT_ENV_VAR.
402
4032021-07-01  Noah Sanci <nsanci@redhat.com>
404
405	PR27711
406	* debuginfod.cxx (options): Add --regex-groom, -r option.
407	(regex_groom): New static bool defaults to false.
408	(parse_opt): Handle 'r' option by setting regex_groom to true.
409	(groom): Introduce and use reg_include and reg_exclude.
410
4112021-07-09  Noah Sanci  <nsanci@redhat.com>
412
413	PR27983
414	* debuginfod-client.c (debuginfod_query_server): As full-length
415	urls are generated with standardized formats, ignore duplicates.
416	Created out1 and changed out2 error gotos. Updated url creation print
417	statements.
418	(globals): Removed url_delim_char, as it was no longer used.
419
4202021-06-18  Mark Wielaard  <mark@klomp.org>
421
422	* debuginfod-client.c (debuginfod_begin): Don't use client if
423	calloc call failed.
424
4252021-06-03  Frank Ch. Eigler <fche@redhat.com>
426
427	PR27863
428	* debuginfod.cxx (unique_set, unique_set_reserver): New classes.
429	(handler_cb): Use them to implement "after-you" queueing.
430
4312021-05-14  Frank Ch. Eigler <fche@redhat.com>
432
433	PR27859
434	* debuginfod-client.c (debuginfod_client): Retain only
435	long-lived multi handle from PR27701 work.
436	(debuginfo_begin,debuginfod_end): ctor/dtor for surviving field only.
437	(debuginfod_query_server): Rework to reuse multi handle only.
438
4392021-04-19  Martin Liska  <mliska@suse.cz>
440
441	* debuginfod-client.c (debuginfod_query_server): Use startswith.
442	(debuginfod_add_http_header): Likewise.
443	* debuginfod.cxx: Likewise.
444
4452021-05-04  Alice Zhang <alizhang@redhat.com>
446
447	* debuginfod-client.c (cache_miss_default_s): New static time_t,
448	defaults to 600 (10 minutes).
449	(cache_miss_filename): New static char pointer.
450	(debuginfod_config_cache): New static function.
451	(debuginfod_clean_cache): Use debuginfod_config_cache for
452	interval_path and max_unused_path.
453	(debuginfod_query_server): Check whether target_cache_path exists
454	as negative cache file and create target_cache_path when the server
455	returns ENOENT. Check cache_miss_path fir cache miss time.
456
4572021-04-26  Frank Ch. Eigler <fche@redhat.com>
458
459	PR27571
460	* debuginfod-client.c (debuginfod_query_server): Chmod 0400 files
461	delivered into the cache to prevent accidental modification.
462
4632021-04-26  Frank Ch. Eigler <fche@redhat.com>
464
465	PR26125
466	* debuginfod-client.c (debuginfod_clean_cache): For directory
467	rmdir, check mtime first.
468	(debuginfod_query_server): Try mkdir / mkstemp up to twice,
469	in case of race.
470
4712021-04-23  Frank Ch. Eigler <fche@redhat.com>
472
473	PR27701
474	* debuginfod-client.c (struct debuginfod_client): Add long-lived
475	CURL easy and multi handles.
476	(debuginfo_begin,debuginfod_end): ctor/dtor for these.
477	(debuginfod_query_server): Rework to reuse easy & multi handles.
478	(*_envvar): Just use the DEBUGINFOD_*_ENV_VAR directly instead.
479
480	* debuginfod.cxx (dc_pool): New pile of reusable debuginfod_client
481	objects for upstream federation connections.
482	(debuginfod_pool_{begin,end,groom}): New functions.
483	(handle_buildid): Use them.
484	(handler_cb): Fix keep-alive given libmicrohttpd convention of multiple
485	callbacks.
486
4872021-04-15  Frank Ch. Eigler <fche@redhat.com>
488
489	* debuginfod.cxx (groom): Only update database stats once.
490
4912021-04-15  Frank Ch. Eigler <fche@redhat.com>
492
493	* debuginfod.cxx (elf_classify): Recognize symtab-only stripped files
494	like fedora's libicudata as debuginfo files.
495
4962021-03-30  Frank Ch. Eigler <fche@redhat.com>
497
498	* debuginfod.cxx (main): Set child thread names.
499
5002021-03-07  Timm Bäder <tbaeder@redhat.com>
501
502	* debuginfod-client.c (debuginfod_query_server): Tweak
503	double/long clamping arithmetic to avoid UB and warnings.
504
5052021-02-25  Frank Ch. Eigler <fche@redhat.com>
506
507	* debuginfod.cxx (handler_cb): Filter webapi for bad
508	artifacttype keywords early for metric hygiene.
509
5102021-02-14  Frank Ch. Eigler <fche@redhat.com>
511
512	* debuginfod.cxx (main -U): Use bsdtar unconditionally.  Also map
513	the debian-compatible .ipk (openembedded distro family) to same.
514
5152021-02-04  Frank Ch. Eigler <fche@redhat.com>
516
517	PR27092 low-memory handling
518	* debuginfod.cxx (fdcache_mintmp): New parameter, with cmd-line option.
519	(parse_opt): Parse it.
520	(main): Default it.
521	(statfs_free_enough_p): New function.
522	(libarchive_fdcache::*): Call it to trigger emergency fdcache flush.
523	(thread_main_scanner): Call it to report filesystem fullness metrics.
524	(groom): Ditto.
525	(set/add_metric): Take double rather than int64_t values.
526	(archive_exception): Propagate suberror to metric label.
527	(main): Detect pthread creation fatal errors properly.
528
5292021-02-02  Frank Ch. Eigler <fche@redhat.com>
530
531	PR27323
532	* debuginfod.cxx (dbq): New read-only database connection for queries
533	only.
534	(signal_handler): Interrupt it.
535	(main): Open / close it.
536	(handle_buildid): Use it for webapi queries only.
537	(database_stats_report): Make more interruptible.  Report sqlite3
538	operation times to the prometheus metrics.
539	(groom): Make more interruptible.
540	(thread_main_fts_source_paths, thread_main_groom): Ensure
541	state/progress metrics are fresh even in case of exceptions.
542
5432020-12-20  Dmitry V. Levin  <ldv@altlinux.org>
544
545	* .gitignore: New file.
546
5472020-12-12  Dmitry V. Levin  <ldv@altlinux.org>
548
549	* debuginfod-client.c (debuginfod_query_server): Fix spelling typos in
550	comments.
551	* debuginfod.cxx: Likewise.
552	(parse_opt): Fix spelling typos in error diagnostics.
553
5542020-12-08  Dmitry V. Levin  <ldv@altlinux.org>
555
556	* Makefile.am [LIBDEBUGINFOD]: Create libdebuginfod.so.1 first, turn
557	libdebuginfod.so into symlink.
558
5592020-11-30  Dmitry V. Levin  <ldv@altlinux.org>
560
561	* Makefile.am (libdebuginfod.so): Replace $@.$(VERSION) with
562	$(LIBDEBUGINFOD_SONAME).
563	(install, uninstall, MOSTLYCLEANFILES): Replace
564	libdebuginfod.so.$(VERSION) with $(LIBDEBUGINFOD_SONAME).
565	(VERSION): Remove.
566	* debuginfod.h: Rename to ...
567	* debuginfod.h.in ... this.
568	(DEBUGINFOD_SONAME): New macro.
569
5702020-11-30  Dmitry V. Levin  <ldv@altlinux.org>
571
572	* Makefile.am (libdebuginfod.so$(EXEEXT)): Drop $(EXEEXT) suffix.
573
5742020-11-25  Frank Ch. Eigler  <fche@redhat.com>
575
576	* debuginfod.cxx (step_ok_done): Correct typo in prom metric label.
577
5782020-11-25  Frank Ch. Eigler  <fche@redhat.com>
579
580	* debuginfod.cxx (tmp_ms_metric): Switch from gettimeofday to
581	clock_gettime(CLOCK_MONOTONIC) for time-interval measurements.
582	(handler_cb, scan_source_paths, groom): Ditto.
583
5842020-11-23  Frank Ch. Eigler  <fche@redhat.com>
585
586	* debuginfod.cxx (tmp_ms_metric): New class for RAII timing metrics.
587	(sqlite_ps::reset, step*): Call it to track sqlite3 performance.
588	(sqlite_exception ctor): Increment sqlite3 error_count.
589
5902020-11-23  Mark Wielaard  <mark@klomp.org>
591
592	* debuginfod-client.c (debuginfod_query_server): Initialize
593	struct handle_data errbuf to the empty string.
594
5952020-11-11  Mark Wielaard  <mark@klomp.org>
596
597	* debuginfod-client.c (debuginfod_set_verbose_fd): New function.
598	(struct debuginfod_client): Add verbose_fd.
599	(struct handle_data): Add errbuf.
600	(debuginfod_query_server): Produce verbose output when
601	debuginfod_client verbose_fd is set. Only clear old data and set
602	default_headers when any work is done. Always goto out when setting
603	rc to an error value. Use CURLOPT_ERRORBUFFER to get more error
604	output when verbose output is requested.
605	* debuginfod.h (DEBUGINFOD_VERBOSE_ENV_VAR): New.
606	(debuginfod_set_verbose_fd): Added.
607	* debuginfod-find.c (parse_opt): Set debuginfod_set_verbose_fd on -v.
608	* bdebuginfod.map (ELFUTILS_0.183): New section, add
609	debuginfod_set_verbose_fd.
610
6112020-11-21  Mark Wielaard  <mark@klomp.org>
612
613	* debuginfod.cxx (handle_root): New function.
614	(handler_cb): Handle "/" and report url1 in webapi error.
615
6162020-11-11  Mark Wielaard  <mark@klomp.org>
617
618	* debuginfod-find.c (progressfn): Use clock_gettime to print Progress
619	at most 5 times a second.
620
6212020-11-19  Frank Ch. Eigler  <fche@redhat.com>
622
623	* debuginfod.cxx (tmp_inc_metric): New class.
624	(handler_cb): Use it to track webapi operations.
625
6262020-11-01  Érico N. Rolim  <erico.erc@gmail.com>
627
628	* debuginfod-client.c (debuginfod_init_cache): Use ACCESSPERMS for
629	mkdir, DEFFILEMODE for open with O_CREAT.
630
6312020-11-01  Érico N. Rolim  <erico.erc@gmail.com>
632
633	* debuginfod.cxx: include libintl.h.
634
6352020-11-01  Érico N. Rolim  <erico.erc@gmail.com>
636
637	* Makefile.am (debuginfod_LDADD): Add argp_LDADD and fts_LIBS.
638	(debuginfod_find_LDADD): Likewise.
639	(libdebuginfod_so_LDLIBS): Add fts_LIBS.
640
6412020-10-31  Frank Ch. Eigler  <fche@redhat.com>
642
643	* debuginfod.cxx (scan_source_file, scan_archive_file): Add new scanned_bytes_total,
644	scanned_files_total metrics.
645	(archive_classify): Exit early if interrupted.
646	(scan_source_paths): Perform realpath/regex checks only on FTS_F files.
647	Tweak metrics.
648
6492020-10-30  Frank Ch. Eigler  <fche@redhat.com>
650
651	PR26775 cont'd.
652	* debuginfod.cxx (thread_main_scanner): Ensure control doesn't
653	leave infinite loop until program exit, even if SIGUSR2.
654	(scan_source_paths): Have traverser clean scanq on
655	SIGUSR2.  Emit additional traversed_total metrics.
656	(groom): Emit additional groomed_total metrics.
657	(thread_main_groom): Restore previous thread_work_total
658	metric.
659
6602020-10-29  Frank Ch. Eigler  <fche@redhat.com>
661
662	PR26775
663	* debuginfod.cxx (forced_*_count): Make these global.
664	(runq::clear): New function.
665	(thread_main_scanner): Check for pending SIGUSR2; interrupt.
666	(scan_source_paths): Check for pending SIGUSR2; interrupt.
667	(groom): Report prometheus stats before groom also.  Check for
668	pending SIGUSR1; interrupt.  Increment thread_work_total for
669	each file scanned, not the entire cycle.
670
6712020-10-29  Frank Ch. Eigler  <fche@redhat.com>
672
673	PR26810
674	* debuginfod.cxx (handle_buildid_*_match): Throw exceptions for
675	more lower level libc errors.
676	(handle_buildid_match): Catch & report exceptions but return 0
677	for continued iteration in the caller.
678
6792020-10-25  Mark Wielaard  <mark@klomp.org>
680
681	* debuginfod-client.c (debuginfod_query_server): Translate
682	CURLE_PEER_FAILED_VERIFICATION to ECONNREFUSED.
683
6842020-10-20  Frank Ch. Eigler  <fche@redhat.com>
685
686	PR26756: more prometheus metrics
687	* debuginfod.cxx (*_exception): Add counters for error occurrences.
688	(fdcache::*): Add counters for fdcache operations and status.
689	(fdcache::set_metric): New fn for overall stat counts.
690	(fdcache::limit): ... allow metric-less use from dtors.
691
6922020-10-20  Frank Ch. Eigler  <fche@redhat.com>
693
694	* debuginfod.cxx (handle_buildid*): Add a parameter for detecting
695	internally-originated lookups for dwz resolution.
696
6972020-09-18  Frank Ch. Eigler <fche@redhat.com>
698
699	* debuginfod.cxx (scan_source_file, archive_classify): Store only
700	canonicalized file names in sdef & sref records in the database.
701
7022020-09-08  Mark Wielaard  <mark@klomp.org>
703
704	* Makefile.am (BUILD_STATIC): Include libcurl_LIBS in libdebuginfod
705	when NOT DUMMY_LIBDEBUGINFOD.
706
7072020-09-16  Mark Wielaard  <mark@klomp.org>
708
709	* debuginfod-find.c: Fix license block comment.
710
7112020-09-15  Mark Wielaard  <mark@klomp.org>
712
713	* debuginfod-find.c (main): Use dwelf_elf_begin.
714
7152020-07-03  Alice Zhang <alizhang@redhat.com>
716
717	* debuginfod-client.c (debuginfod_query_server): Use strncasecmp
718	to compare effective_url. Try CURLINFO_SCHEME as fallback.
719
7202020-06-19  Mark Wielaard  <mark@klomp.org>
721
722	* Makefile.am (bin_PROGRAMS): Guard with DEBUGINFOD and
723	LIBDEBUGINFOD.
724	(debuginfod_LDADD): Remove libcurl.
725	(libdebuginfod): When static and DUMMY_LIBDEBUGINFO remove libcurl.
726	(noinst_LIBRARIES): Guard with LIBDEBUGINFOD.
727	(AM_CPPFLAGS): Add -Wno-unused-parameter when DUMMY_LIBDEBUGINFOD.
728	(pkginclude_headers): Guard with LIBDEBUGINFOD
729	(libdebuginfod_so_LIBS): Likewise.
730	(+libdebuginfod_so_LDLIBS): Likewise.
731	(install): Likewise.
732	(uninstall): Likewise.
733	* debuginfod-client.c: Include dummy functions when
734	DUMMY_LIBDEBUGINFOD.
735	* debuginfod.cxx: Remove curl.h include.
736
7372020-06-16  Mark Wielaard  <mark@klomp.org>
738
739	* debuginfod-client.c (debuginfod_query_server): Check malloc.
740	Move curl_multi_init call before handle_data malloc call.
741
7422020-06-16  Mark Wielaard  <mark@klomp.org>
743
744	* debuginfod-client.c (debuginfod_query_server): Replace sizeof
745	build_id_bytes check with strlen build_id check.
746
7472020-06-16  Mark Wielaard  <mark@klomp.org>
748
749	* debuginfod-client.c (debuginfod_query_server): Increase suffix
750	array and prepare having to escape 1 character with 2.
751
7522020-06-16  Mark Wielaard  <mark@klomp.org>
753
754	* debuginfod-client.c (debuginfod_clean_cache): Handle failing
755	fopen (interval_path).
756
7572020-03-29  Mark Wielaard  <mark@klomp.org>
758
759	* debuginfod-client.c (debuginfod_add_http_header): Check header
760	contains precisely one colon that isn't the first or last char.
761
7622020-03-29  Frank Ch. Eigler  <fche@redhat.com>
763
764	* debuginfod-client.c (struct debuginfod_client): Add a flag field
765	for progressfn printing.
766	(default_progressfn): Set it if printing \rsomething.
767	(debuginfod_end): Terminate with \n if flag set, i.e., only if the
768	default_progressfn was actually called.
769
7702020-03-27  Mark Wielaard  <mark@klomp.org>
771
772	* debuginfod.cxx (parse_opt): Check port is not zero.
773
7742020-03-28  Frank Ch. Eigler  <fche@redhat.com>
775
776	* debuginfod.cxx (handle_buildid_r_match): During archive
777	extraction / fdcache prefetching, set the mtime of each
778	file in the cache.
779
7802020-03-27  Frank Ch. Eigler  <fche@redhat.com>
781
782	* debuginfod-find.c (main): Extract buildid from /binary/ if
783	given instead of hex string.
784	* Makefile.am: Add elfutils library prereqs for debuginfod-find.
785
7862020-03-24  Frank Ch. Eigler  <fche@redhat.com>
787
788	* debuginfod.h, libdebuginfod.map: New functions for _add_url_header.
789	* debuginfod-client.c (struct debuginfod_client): Add headers fields.
790	(debuginfod_add_http_header): New client api to add outgoing headers.
791	(add_default_headers): Renamed from add_extra_headers, skip if flag.
792	(debuginfod_query_server): Pass accumulated headers to libcurl.
793	(debuginfod_end): Clean accumulated headers.
794	(debuginfod_find_*): Add default headers at this point.
795	* debuginfod.cxx (handle_buildid): Add conn pointer.  Use it to relay
796	incoming UA and XFF headers to federated upstream debuginfods.
797
7982020-03-26  Frank Ch. Eigler <fche@redhat.com>
799
800	* debuginfod.cxx (handler_cb): Export two families of metrics for
801	prometheus traffic analysis: response times and data amounts.
802
8032020-03-26  Frank Ch. Eigler <fche@redhat.com>
804
805	* debuginfod.cxx (parse_opt): For -U, prefer dpkg-deb
806	after all if access(3)-able, fallback to bsdtar.
807
8082020-03-25  Frank Ch. Eigler <fche@redhat.com>
809
810	* debuginfod.cxx (parse_opt): Associate a bsdtar subshell with
811	the .deb & .ddeb extensions, instead of dpkg-deb.
812
8132020-03-26  Frank Ch. Eigler  <fche@redhat.com>
814
815	* debuginfod-client.c (debuginfod_query_server): Don't
816	set CURLOPT_PATH_AS_IS on old curl.  Mostly harmless.
817
8182020-03-24  Frank Ch. Eigler  <fche@redhat.com>
819
820	* debuginfod-client.c (debuginfod_query_server): Set
821	CURLOPT_PATH_AS_IS, to propagate file names verbatim.
822	* debuginfod.cxx (canon_pathname): Implement RFC3986
823	style pathname canonicalization.
824	(handle_buildid): Canonicalize incoming webapi source
825	paths, accept either one.
826	(scan_source_file, archive_classify): Store both
827	original and canonicalized dwarf-source file names.
828
8292020-03-24  Frank Ch. Eigler  <fche@redhat.com>
830
831	* debuginfod.cxx (handle_buildid): In case of federated fallback
832	queries, handle errors analogously to local ENOENT/404.
833	(handle_metrics): Return a size-of-response value.
834	(handler_cb): Add code to time entire application-side processing
835	stage + response sizes + http codes, so as to emit a complete
836	httpd-flavoured log line for each webapi request.
837
8382020-03-24  Frank Ch. Eigler  <fche@redhat.com>
839
840	* debuginfod-client.c (debuginfod_query_server): Print the
841	default_progressfn terminating \n message only if that progressfn
842	is actually set.
843
8442020-03-24  Frank Ch. Eigler  <fche@redhat.com>
845
846	* debuginfod-find.c (main): Correct /source full-pathness check for
847	"debuginfod-find -v source deadbeef /pathname" case.
848
8492020-03-22  Frank Ch. Eigler  <fche@redhat.com>
850
851	* debuginfod-client.c (struct debuginfod_client): Add url field.
852	(struct handle_data): Add client field as backpointer.
853	(debuginfod_write_callback): Compute & save URL.
854	(default_progressfn): Print front pieces of the URL.
855	(debuginfod_query_server): Clear URL and cleanup after progressfn.
856	* debuginfod-find.c (main): Print URL at transfer conclusion.
857
8582020-03-22  Frank Ch. Eigler  <fche@redhat.com>
859
860	* debuginfod.h, libdebuginfod.map: New functions for _get/set_user().
861	* debuginfod-client.c: Implement them.
862	* debuginfod-find.c: Include a token call just for testing them.
863
8642020-03-03  Aaron Merey  <amerey@redhat.com>
865
866	* debuginfod-client.c (debuginfod_query_server): Update
867	cache_path even when new default path already exists.
868
8692020-02-27  Aaron Merey  <amerey@redhat.com>
870
871	* debuginfod-client.c (xalloc_str): New macro. Call
872	asprintf with error checking.
873	(debuginfod_query_server): Use XDG_CACHE_HOME as a default
874	cache location if it is set. Replace snprintf with xalloc_str.
875
8762020-02-26  Konrad Kleine <kkleine@redhat.com>
877
878	* debuginfod-client.c (debuginfod_query_server): Handle curl's
879	response code correctly when DEBUGINFOD_URLS begin with file://
880
8812020-02-25  Frank Ch. Eigler  <fche@redhat.com>
882
883	* debuginfod.cxx (parse_opt): Treat -R as if -Z.rpm .
884
8852020-02-25  Frank Ch. Eigler  <fche@redhat.com>
886
887	* debuginfod.cxx (fdcache_prefetch): New parameter.
888	(parse_opt): Parse it.
889	(main): Default it.
890	(fdcache::fd_size_mb): Change to double for accuracy.
891	(fdcache::probe): New function.
892	(fdcache::intern): New option to intern at end of LRU.
893	(fdcache::lookup): Clean fdcache.
894	(handle_buildid_r_match): Implement multi-stage archive
895	parsing, with optional prefetching of extracted contents
896	into the fdcache.
897
8982020-02-19  Aaron Merey  <amerey@redhat.com>
899
900	* debuginfod-client.c (debuginfod_clean_cache): Restrict
901	cleanup to client-pattern files.
902
9032020-02-05  Frank Ch. Eigler  <fche@redhat.com>
904
905	* debuginfod.cxx (argp options): Add -Z option.
906	(canonicalized_archive_entry_pathname): New function for
907	distro-agnostic file name matching/storage.
908
9092020-01-22  Frank Ch. Eigler  <fche@redhat.com>
910
911	* debuginfod.cxx (dwarf_extract_source_paths): Don't print
912	"skipping hat" messages at verbosity <=3, too noisy.
913
9142020-01-19  Frank Ch. Eigler  <fche@redhat.com>
915
916	* debuginfod.cxx (scanq): Rework to let groomer/fts threads
917	synchronize with an empty workqueue, and lock out workqueue
918	consumers.
919	(thread_groom): Adopt new scanq idle APIs to lock out scanners.
920	(thread_main_fts_source_paths): Adopt new scanq idler API to
921	avoid being restarted while scanners haven't even finished yet.
922	(thread_main_*): Increment thread_work_total metric only after
923	a work cycle is completed, not when it begins.
924
9252020-01-18  Frank Ch. Eigler  <fche@redhat.com>
926
927	* debuginfod.cxx (thread_main_scanner): Handle empty source_paths[].
928
9292020-01-11  Frank Ch. Eigler  <fche@redhat.com>
930
931	* debuginfod.cxx (libarchive_fdcache): New class/facility to own a
932	cache of temporary files that were previously extracted from an
933	archive.  If only it could store just unlinked fd's instead of
934	filenames.
935	(handle_buildid_r_match): Use it to answer dwz/altdebug and webapi
936	requests.
937	(groom): Clean it.
938	(main): Initialize the cache control parameters from heuristics.
939	Use a consistent tmpdir for these and tmp files elsewhere.
940
9412020-01-11  Frank Ch. Eigler  <fche@redhat.com>
942
943	* debuginfod.cxx (conninfo): Print User-Agent and X-Forwarded-For
944	request headers, after mild safety-censorship (for easier machine
945	processing).
946
9472020-01-11  Frank Ch. Eigler  <fche@redhat.com>
948
949	* debuginfod.cxx: Rework threading model.
950	(workq): New class for concurrent work-queue.
951	(semaphore): Removed class, now unused.
952	(scan_source_file_path): Rework into ...
953	(scan_source_file): New function.
954	(thread_main_scan_source_file_path): Nuke.
955	(scan_source_archive_path): Rework into ...
956	(scan_archive_file): New function.
957	(thread_main_scanner): New function for scanner threads.
958	(thread_main_fts_source_paths): New function for traversal thread.
959	(scan_source_paths): ... doing this.
960	(thread_groom): Tweak metrics for consistency.
961	(main): Start 1 traversal and N scanner threads if needed.
962
9632019-01-02  Mark Wielaard  <mark@klomp.org>
964
965	* debuginfod.cxx (default_connect_timeout): Removed.
966	(default_transfer_timeout): Removed.
967	(default_timeout): New. Default to 90 seconds.
968	(debuginfod_query_server): Parse server_timeout_envvar as one number.
969	Set as CURLOPT_LOW_SPEED_TIME, with CURL_OPT_LOW_SPEED_LIMITE as 100K.
970
9712020-01-09  Frank Ch. Eigler  <fche@redhat.com>
972
973	* debuginfod-client.c (add_extra_headers): New function,
974	based on mjw's draft.
975	(debuginfod_query_server): Call it.
976
9772019-12-22  Frank Ch. Eigler  <fche@redhat.com>
978
979	* debuginfod.cxx (*_rpm_*): Rename to *_archive_* throughout.
980	(scan_archives): New read-mostly global to identify archive
981	file extensions and corresponding extractor commands.
982	(parse_opt): Handle new -U flag.
983
9842019-12-19  Frank Ch. Eigler  <fche@redhat.com>
985
986	* debuginfod-client.c (default_progressfn): New function.
987	(debuginfod_begin): Use it if $DEBUGINFOD_PROGRESS set.
988	(server_timeout): Bump to 30 seconds.
989	(debuginfod_query_server): Call progressfn -after- rather than
990	before curl ops, to make it likely that a successful transfer
991	results in final a=b call.  Tweak cleanup sequence.
992	* debuginfod.h: Document $DEBUGINFOD_PROGRESS name.
993
9942019-12-09  Mark Wielaard  <mark@klomp.org>
995
996	* debuginfod-client.c (debuginfod_query_server): Check
997	server_urls_envvar early.
998
9992019-12-03  Mark Wielaard  <mark@klomp.org>
1000
1001	* debuginfod-client.c (debuginfod_query_server): Use separate
1002	local variables for CURLcode curl_res and CURLMcode curlm_res.
1003
10042019-11-26  Mark Wielaard  <mark@klomp.org>
1005
1006	* Makefile.am (BUILD_STATIC): Add needed libraries for libdw and
1007	libdebuginfod.
1008
10092019-11-25  Frank Ch. Eigler  <fche@redhat.com>
1010
1011	* debuginfod.cxx (groom): Add a sqlite3_db_release_memory()
1012	at the end of periodic grooming to try to shrink the process.
1013
10142019-11-24  Mark Wielaard  <mark@klomp.org>
1015
1016	* debuginfod.cxx (test_webapi_sleep): Removed.
1017	(handler_cb): Don't check test_webapi_sleep and sleep.
1018	(main): Don't set test_webapi_sleep.
1019
10202019-11-24  Mark Wielaard  <mark@klomp.org>
1021
1022	* debuginfod.cxx (add_metric): New function.
1023	(scan_source_file_path): Record metrics for
1024	found_executable_total, found_debuginfo_total and
1025	found_sourcerefs_total.
1026	(scan_source_rpm_path): Likewise.
1027
10282019-11-07  Frank Ch. Eigler  <fche@redhat.com>
1029
1030	* debuginfod.cxx: Add /metrics endpoint.  Add numerous
1031	calls to new functions inc_metric/set_metric to populate
1032	threadsafe map containing stats.  Add http content-type
1033	response headers throughout.
1034	(thread_main_*): Simplify counter/timer flow.
1035	(main): Reorder web service shutdown to leave http running
1036	as long as possible.
1037	* debuginfod.8: Document it, add security caution.
1038
10392019-11-06  Frank Ch. Eigler  <fche@redhat.com>
1040
1041	* debuginfod.cxx: Add new -L (symlink-following) mode.
1042	* debuginfod.8: Document it.
1043
10442019-11-04  Frank Ch. Eigler  <fche@redhat.com>
1045
1046	* debuginfo-client.c (debuginfod_set_progressfn): New function
1047	for progress/interrupt callback.
1048	(debuginfod_clean_cache, debuginfod_query_server): Call it.
1049	* debuginfo.h: Declare it.
1050	* debuginfod_set_progressfn.3, *_find_debuginfo.3: Document it.
1051	* Makefile.am: Install it.
1052	* libdebuginfod.map: Export it all under ELFUTILS_0.178 symversion.
1053
1054	* debuginfod-find.c: Add -v option to activate progress cb.
1055	* debuginfod-find.1: Document it.
1056	* debuginfod.cxx: Add $DEBUGINFOD_TEST_WEBAPI_SLEEP env var
1057	to insert sleep in webapi callbacks, to help manual testing.
1058
10592019-10-28  Frank Ch. Eigler  <fche@redhat.com>
1060
1061	* debuginfod.cxx: New file: debuginfod server.
1062	* debuginfod.8: New file: man page.
1063	* Makefile.am: Build it.
1064
10652019-10-28  Aaron Merey  <amerey@redhat.com>
1066
1067	* debuginfod-client.c: New file: debuginfod client library.
1068	* debuginfod.h: New file: header for same.
1069	* libdebuginfod.map: New file: govern its solib exports.
1070	* debuginfod-find.c: New file: command line frontend.
1071	* debuginfod-find.1, debuginfod_find_source.3,
1072	debuginfod_find_executable.3, debuginfod_find_debuginfo.3:
1073	New man pages.
1074