Lines Matching full:the
3 # Copyright 2016 The Chromium OS Authors. All rights reserved.
5 # found in the LICENSE file.
6 """Processes the functions from the pprof(go/pprof) files and CWP(go/cwp) data.
8 The pprof --top and pprof --tree outputs should be extracted from the benchmark
9 profiles. The outputs contain the hot functions and the call chains.
11 For each pair of pprof --top and --tree output files, the tool will create a
12 file that contains the hot functions present also in the extracted CWP data.
13 The common functions are organized in groups that represent a Chrome OS
20 Afterwards, this script extracts the functions that are present in the CWP
21 data and not in the benchmark profiles. The extra functions are also groupped
37 """Does the pprof and CWP output processing.
39 Extracts the common, extra functions from the pprof files, groups them in
40 Chrome OS components. Computes the metrics for the common functions,
50 """Initializes the HotFunctionsProcessor.
53 pprof_top_path: The directory containing the files with the pprof --top
55 pprof_tree_path: The directory containing the files with the pprof --tree
57 cwp_inclusive_count_file: The CSV file containing the CWP functions with
58 the inclusive count values.
59 cwp_pairwise_inclusive_count_file: The CSV file containing the CWP pairs
61 cwp_function_groups_file: The file that contains the CWP function groups.
62 common_functions_path: The directory containing the CSV output files
63 with the common functions of the benchmark profiles and CWP data.
64 common_functions_groups_path: The directory containing the CSV output
65 files with the CWP groups and their metrics that match the common
66 functions of the benchmark profiles and CWP.
67 benchmark_set_metrics_file: The CSV output file containing the metrics for
69 extra_cwp_functions_file: The CSV output file containing the functions
70 that are in the CWP data, but are not in any of the benchmark profiles.
71 extra_cwp_functions_groups_file: The CSV output file containing the groups
72 that match the extra CWP functions and their statistics.
73 extra_cwp_functions_groups_path: The directory containing the CSV output
74 files with the extra CWP functions that match a particular group.
89 """Does the processing of the hot functions."""
112 """Extracts the common functions of the benchmark profiles and the CWP data.
115 file with the same name containing the common functions specifications and
116 metrics, that will be placed in the common_functions_path directory.
118 The resulting file is in CSV format, containing the following fields:
122 For each pair of pprof files, an additional file is created with the
123 Chrome OS groups that match the common functions.
125 The file is in CSV format containing the fields: group name, group path,
126 the number of functions that match the group, the average and cumulative
127 distance, the average and cumulative score.
128 The file has the same name with the pprof file and it is placed in the
131 For all the analyzed benchmarks, the method creates a CSV output file
132 containing the metrics for each benchmark. The CSV fields include the
133 benchmark name, the number of common functions, the average and
136 It builds a dict of the CWP statistics by calling the
141 pprof_top_path: The name of the directory with the files with the
143 pprof_tree_path: The name of the directory with the files with the
145 cwp_inclusive_count_file: A dict with the inclusive count values.
146 cwp_pairwise_inclusive_count_file: A dict with the pairwise inclusive
148 cwp_function_groups: A list of tuples containing the name of the group
149 and the corresponding file path.
150 common_functions_path: The path containing the output files with the
152 common_functions_groups_path: The path containing the output files with
153 the Chrome OS groups that match the common functions and their metrics.
154 benchmark_set_metrics_file: The CSV output file containing the metrics for
155 all the analyzed benchmarks.
158 A dict containing the CWP statistics with the common functions marked as
254 """Groups the extra functions.
256 Writes the data of the functions that belong to each group in a separate
257 file, sorted by their inclusive count value, in descending order. The file
258 name is the same as the group name.
260 The file is in CSV format, containing the fields: function name, file name,
263 It creates a CSV file containing the name of the group, their
264 common path, the total inclusive count and inclusive count fraction values
265 of all the functions declared in files that share the common path, sorted
266 in descending order by the inclusive count value.
269 cwp_statistics: A dict containing the CWP statistics.
270 cwp_function_groups: A list of tuples with the groups names and the path
271 describing the groups.
272 extra_cwp_functions_groups_path: The name of the directory containing
273 the CSV output files with the extra CWP functions that match a
275 extra_cwp_functions_groups_file: The CSV output file containing the groups
276 that match the extra functions and their statistics.
328 """Gets the functions that are in the CWP data, but not in the pprof output.
330 Writes the functions and their statistics in the extra_cwp_functions_file
331 file. The output is sorted based on the inclusive_count value. The file is
332 in CSV format, containing the fields: function name, file name, object name,
336 cwp_statistics: A dict containing the CWP statistics indexed by the
337 function and the file name, comma separated.
338 extra_cwp_functions_file: The file where it should be stored the CWP
361 help='The directory containing the files with the pprof --top output of '
362 'the benchmark profiles (the hot functions). The name of the files '
363 'should match with the ones from the pprof tree output files.')
367 help='The directory containing the files with the pprof --tree output '
368 'of the benchmark profiles (the call chains). The name of the files '
369 'should match with the ones of the pprof top output files.')
373 help='The CSV file containing the CWP hot functions with their '
374 'inclusive_count values. The CSV fields include the name of the '
375 'function, the file and the object with the definition, the inclusive '
376 'count value and the inclusive count fraction out of the total amount of '
381 help='The CSV file containing the CWP pairs of parent and child '
382 'functions with their inclusive count values. The CSV fields include the '
383 'name of the parent and child functions concatenated by ;;, the file '
384 'and the object with the definition of the child function, and the '
389 help='The file that contains the CWP function groups. A line consists in '
390 'the group name and a file path describing the group. A group must '
395 help='The directory containing the CSV output files with the common '
396 'functions of the benchmark profiles and CWP data. A file will contain '
397 'all the hot functions from a pprof top output file that are also '
398 'included in the file containing the cwp inclusive count values. The CSV '
399 'fields are: the function name, the file and the object where the '
400 'function is declared, the CWP inclusive count and inclusive count '
401 'fraction values, the cumulative and average distance, the cumulative '
402 'and average score. The files with the common functions will have the '
403 'same names with the corresponding pprof output files.')
407 help='The directory containing the CSV output files with the Chrome OS '
408 'groups and their metrics that match the common functions of the '
409 'benchmark profiles and CWP. The files with the groups will have the '
410 'same names with the corresponding pprof output files. The CSV fields '
411 'include the group name, group path, the number of functions that match '
412 'the group, the average and cumulative distance, the average and '
417 help='The CSV output file containing the metrics for each benchmark. The '
418 'CSV fields include the benchmark name, the number of common functions, '
419 'the average and cumulative distance and score.')
423 help='The CSV output file containing the functions that are in the CWP '
424 'data, but are not in any of the benchmark profiles. The CSV fields '
425 'include the name of the function, the file name and the object with the '
426 'definition, and the CWP inclusive count and inclusive count fraction '
427 'values. The entries are sorted in descending order based on the '
432 help='The CSV output file containing the groups that match the extra CWP '
433 'functions and their statistics. The CSV fields include the group name, '
434 'the file path, the total inclusive count and inclusive count fraction '
435 'values of the functions matching a particular group.')
439 help='The directory containing the CSV output files with the extra CWP '
440 'functions that match a particular group. The name of the file is the '
441 'same as the group name. The CSV fields include the name of the '
442 'function, the file name and the object with the definition, and the CWP '
443 'inclusive count and inclusive count fraction values. The entries are '
444 'sorted in descending order based on the inclusive count value.')