88 lines
3.8 KiB
Python
Raw Normal View History

This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
#!/usr/bin/env python
# Copyright (c) 2012 The WebRTC project authors. All Rights Reserved.
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
#
# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS. All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.
'''Runs various WebRTC tests through valgrind_test.py.
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
This script inherits the chrome_tests.py in Chrome, replacing its tests. We do
this by taking chrome's faux cmdline test and making that the standard, so that
we effectively can pass in any binary we feel like. It's also possible to pass
arguments to the test, provided that the arguments do not contain dashes (these
can be "escaped" by passing + instead, so -a becomes +a, and --my-option becomes
++my_option).
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
'''
import optparse
import sys
import logging_utils
import chrome_tests
def _main(_):
parser = optparse.OptionParser("usage: %prog -b <dir> -t <test> "
"[-t <test> ...] <arguments to all tests>"
"NOTE: when passing arguments to all tests, "
" replace any - with +.")
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
parser.add_option("-b", "--build_dir",
help="the location of the compiler output")
parser.add_option("-t", "--test", action="append", default=[],
help="which test to run, supports test:gtest_filter format "
"as well.")
parser.add_option("", "--baseline", action="store_true", default=False,
help="generate baseline data instead of validating")
parser.add_option("", "--gtest_filter",
help="additional arguments to --gtest_filter")
parser.add_option("", "--gtest_repeat",
help="argument for --gtest_repeat")
parser.add_option("-v", "--verbose", action="store_true", default=False,
help="verbose output - enable debug log messages")
parser.add_option("", "--tool", dest="valgrind_tool", default="memcheck",
help="specify a valgrind tool to run the tests under")
parser.add_option("", "--tool_flags", dest="valgrind_tool_flags", default="",
help="specify custom flags for the selected valgrind tool")
parser.add_option("", "--keep_logs", action="store_true", default=False,
help="store memory tool logs in the <tool>.logs directory "
"instead of /tmp.\nThis can be useful for tool "
"developers/maintainers.\nPlease note that the <tool>"
".logs directory will be clobbered on tool startup.")
options, args = parser.parse_args()
if options.verbose:
logging_utils.config_root(logging.DEBUG)
else:
logging_utils.config_root()
if not options.test:
parser.error("--test not specified")
if len(options.test) != 1 and options.gtest_filter:
parser.error("--gtest_filter and multiple tests don't make sense together")
# Performs the deferred-argument black magic described in the usage.
translated_args = map(lambda arg: arg.replace('+', '-'), args)
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
for t in options.test:
tests = chrome_tests.ChromeTests(options, translated_args, t)
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
ret = tests.Run()
if ret: return ret
return 0
if __name__ == "__main__":
# Overwrite the ChromeTests tests dictionary. The cmdline option allows the
# user to pass any executable as parameter to the test script, so we'll use
# that to get our binaries in (hackish but convenient).
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
chrome_tests.ChromeTests._test_list = {
"cmdline": chrome_tests.ChromeTests.RunCmdLine,
}
# We do this so the user can write -t <binary> instead of -t cmdline <binary>.
sys.argv.insert(sys.argv.index('-t') + 1, 'cmdline')
print sys.argv
This CL introduces a new directory for WebRTC utility scripts for Valgrind: tools/valgrind-webrtc To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests: - webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead. - webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script. The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script. The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability. Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily. The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform. The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests. Example: Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL. BUG= TEST= Review URL: http://webrtc-codereview.appspot.com/322010 git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
2011-12-20 08:53:41 +00:00
ret = _main(sys.argv)
sys.exit(ret)