pax_global_header00006660000000000000000000000064143261230150014507gustar00rootroot0000000000000052 comment=fd92c7868f938f24fda1d081d0d416a10a212f42 golang-github-intel-goresctrl-0.3.0/000077500000000000000000000000001432612301500173515ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/.github/000077500000000000000000000000001432612301500207115ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/.github/workflows/000077500000000000000000000000001432612301500227465ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/.github/workflows/verify.yml000066400000000000000000000010541432612301500247750ustar00rootroot00000000000000name: Verify on: [push, pull_request] jobs: build: name: Build runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - uses: actions/setup-go@v2 with: go-version: 1.18 - name: Install golangci-lint run: | curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.49.0 - name: Verify run: make verify - name: Test run: make test - name: Codecov report run: bash <(curl -s https://codecov.io/bash) golang-github-intel-goresctrl-0.3.0/LICENSE000066400000000000000000000261351432612301500203650ustar00rootroot00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. golang-github-intel-goresctrl-0.3.0/Makefile000066400000000000000000000005201432612301500210060ustar00rootroot00000000000000GO_CMD := go Q := @ .PHONY: all ci-lint gofmt-verify test verify all: test verify: gofmt-verify ci-lint gofmt-verify: @out=`gofmt -w -l -d $$(find . -name '*.go')`; \ if [ -n "$$out" ]; then \ echo "$$out"; \ exit 1; \ fi ci-lint: golangci-lint run test: $(Q)$(GO_CMD) test -v -coverprofile=coverage.txt ./pkg/... golang-github-intel-goresctrl-0.3.0/README.md000066400000000000000000000006431432612301500206330ustar00rootroot00000000000000# Go Resource Control The goresctrl library provides Go interface to manage following resources. - CPU cache allocation and memory bandwidth, see the [rdt](doc/rdt.md) (Intel Resource Director Technology) package. - CPU frequency in core granularity, see the [sst](doc/sst.md) (Intel Speed Select Technology) package. - Storage I/O scheduler priority and bandwidth, see the [blockio](doc/blockio.md) package. golang-github-intel-goresctrl-0.3.0/cmd/000077500000000000000000000000001432612301500201145ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/cmd/blockio/000077500000000000000000000000001432612301500215365ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/cmd/blockio/main.go000066400000000000000000000053721432612301500230200ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // This application demonstrates using the blockio API. package main import ( "encoding/json" "flag" "fmt" "os" "github.com/intel/goresctrl/pkg/blockio" ) var examples string = `Examples: # Inspect OCI blockio structure $ blockio -config sample.cfg -class slowread | jq # Apply read throttling to a cgroup $ blockio -config sample.cfg -class slowread -cgroup user.slice/mygroup $ cat /sys/fs/cgroup/blkio/user.slice/mygroup/blkio.throttle.read_bps_device # Remove throttling from a cgroup $ blockio -config sample.cfg -class nolimit -cgroup user.slice/mygroup ` func usage() { flag.CommandLine.SetOutput(os.Stdout) fmt.Fprintln(flag.CommandLine.Output(), "blockio - demo application for goresctrl/pkg/blockio API") fmt.Fprintln(flag.CommandLine.Output(), "Usage: blockio -config=FILE -class=NAME [-cgroup=CGROUP]") flag.PrintDefaults() fmt.Fprint(flag.CommandLine.Output(), examples) } func error(format string, args ...interface{}) { fmt.Fprintln(os.Stderr, fmt.Sprintf(format, args...)) os.Exit(1) } func main() { // Parse commandline arguments flag.Usage = usage optConfig := flag.String("config", "", "load class configuration from FILE") optClass := flag.String("class", "", "use configuration of the blockio class NAME") optCgroup := flag.String("cgroup", "", "apply class to CGROUP, otherwise print it as OCI BlockIO structure") flag.Parse() if optConfig == nil || *optConfig == "" { error("missing -config=FILE") } if optClass == nil || *optClass == "" { error("missing -class=NAME") } // Read blockio class configuration. if err := blockio.SetConfigFromFile(*optConfig, true); err != nil { error("%v", err) } if optCgroup == nil || *optCgroup == "" { // If -cgroup=CGROUP is missing, print OCI spec. oci, err := blockio.OciLinuxBlockIO(*optClass) if err != nil { error("%v", err) } ociBytes, err := json.Marshal(oci) if err != nil { error("%v", err) } fmt.Printf("%s\n", ociBytes) } else { // If -cgroup=CGROUP is given, apply class configuration to it. err := blockio.SetCgroupClass(*optCgroup, *optClass) if err != nil { error("%v", err) } fmt.Printf("cgroup %s configured to blockio class %q\n", *optCgroup, *optClass) } } golang-github-intel-goresctrl-0.3.0/cmd/blockio/sample.cfg000066400000000000000000000002021432612301500234720ustar00rootroot00000000000000classes: slowread: - devices: - /dev/[hsv]d[a-z] throttlereadbps: 5M highprio: - weight: 400 nolimit: golang-github-intel-goresctrl-0.3.0/cmd/rdt/000077500000000000000000000000001432612301500207055ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/cmd/rdt/main.go000066400000000000000000000063661432612301500221730ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // This application demonstrates using the blockio API. package main import ( "flag" "fmt" "os" "sort" "strings" "github.com/intel/goresctrl/pkg/rdt" ) var ( // Global command line flags groupPrefix string ) type subCmd func([]string) error var subCmds = map[string]subCmd{ "info": subCmdInfo, "configure": subCmdConfigure, } func main() { cmds := make([]string, 0, len(subCmds)) for c := range subCmds { cmds = append(cmds, c) } sort.Strings(cmds) allCmds := strings.Join(cmds, ", ") if len(os.Args) < 2 { exitError("missing sub-command, must be one of: %s\n", allCmds) } // Run sub-command cmd, ok := subCmds[os.Args[1]] if !ok { exitError("unknown sub-command %q, must be of: %s\n", os.Args[1], allCmds) } if err := cmd(os.Args[2:]); err != nil { exitError("sub-command %q failed: %v\n", os.Args[1], err) } } func addGlobalFlags(flagset *flag.FlagSet) { flagset.StringVar(&groupPrefix, "group-prefix", "", "prefix to use for resctrl groups") } func subCmdInfo(args []string) error { // Parse command line args flags := flag.NewFlagSet("info", flag.ExitOnError) addGlobalFlags(flags) if err := flags.Parse(args); err != nil { return err } // Run sub-command if err := rdt.Initialize(groupPrefix); err != nil { fmt.Printf("RDT is not enabled: %v\n", err) return nil } fmt.Printf("Monitoring supported: %v\n", rdt.MonSupported()) if rdt.MonSupported() { mon := rdt.GetMonFeatures() fmt.Println("Monitoring features:") for r, f := range mon { fmt.Printf(" - %s: %v\n", r, strings.Join(f, ", ")) } } fmt.Println("Classes:") for _, cls := range rdt.GetClasses() { fmt.Printf(" - %s\n", cls.Name()) mon := cls.GetMonGroups() if len(mon) > 0 { fmt.Println(" Monitoring groups:") for _, grp := range mon { fmt.Printf(" - %s\n", grp.Name()) } } } return nil } func subCmdConfigure(args []string) error { // Parse command line args flags := flag.NewFlagSet("configure", flag.ExitOnError) addGlobalFlags(flags) configFile := flags.String("config-file", "", "path to rdt configuration file") force := flags.Bool("force", false, "force configuration, delete non-empty resctrl groups") if err := flags.Parse(args); err != nil { return err } if *configFile == "" { return fmt.Errorf("-config-file must be specified") } // Run sub-command if err := rdt.Initialize(groupPrefix); err != nil { return fmt.Errorf("RDT is not enabled: %v", err) } fmt.Println("Configuring resctrl filesystem...") if err := rdt.SetConfigFromFile(*configFile, *force); err != nil { return err } fmt.Println("Done!") return nil } func exitError(format string, args ...interface{}) { fmt.Printf("ERROR: "+format+"\n", args...) os.Exit(1) } golang-github-intel-goresctrl-0.3.0/cmd/rdt/sample-conf.yaml000066400000000000000000000010741432612301500237770ustar00rootroot00000000000000options: l2: optional: true l3: optional: true mb: optional: true partitions: default: l2Allocation: "100%" l3Allocation: "100%" mbAllocation: ["100%", "4000000000MBps"] classes: Guaranteed: l3Allocation: "100%" l2Allocation: "100%" mbAllocation: ["100%", "4000000000MBps"] Burstable: l3Allocation: "60%" l2Allocation: "60%" mbAllocation: ["60%", "10000MBps"] Besteffort: l3Allocation: "30%" l2Allocation: "30%" mbAllocation: ["30%", "4000MBps"] golang-github-intel-goresctrl-0.3.0/cmd/sst-ctl/000077500000000000000000000000001432612301500215055ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/cmd/sst-ctl/main.go000066400000000000000000000230021432612301500227550ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package main import ( "flag" "fmt" "os" "sort" "strconv" "strings" "github.com/intel/goresctrl/pkg/sst" "github.com/intel/goresctrl/pkg/utils" ) var ( // Global command line flags packageIds string ) type subCmd func([]string) error var subCmds = map[string]subCmd{ "info": subCmdInfo, "bf": subCmdBF, "cp": subCmdCP, } func main() { cmds := make([]string, 0, len(subCmds)) for c := range subCmds { cmds = append(cmds, c) } sort.Strings(cmds) allCmds := strings.Join(cmds, ", ") if len(os.Args) < 2 { fmt.Printf("missing sub-command, must be one of: %s\n", allCmds) os.Exit(1) } // Run sub-command cmd, ok := subCmds[os.Args[1]] if !ok { fmt.Printf("unknown sub-command %q, must be of: %s\n", os.Args[1], allCmds) os.Exit(1) } if err := cmd(os.Args[2:]); err != nil { fmt.Printf("ERROR: sub-command %q failed: %v\n", os.Args[1], err) os.Exit(1) } } func addGlobalFlags(flagset *flag.FlagSet) { flagset.StringVar(&packageIds, "package", "", "One or more physical package id") } func printPackageInfo(pkgId ...int) error { info, err := sst.GetPackageInfo(pkgId...) if err != nil { return err } fmt.Println(utils.DumpJSON(info)) return nil } // TODO: Move this functionality into utils.NewIdSetFromString() func str2slice(str string) []int { var s []int for _, str := range strings.Split(str, ",") { if str == "" { continue } id, err := strconv.ParseUint(str, 10, 0) if err != nil { fmt.Printf("invalid value '%s': %v", str, err) continue } s = append(s, int(id)) } return s } func subCmdInfo(args []string) error { // Parse command line args flags := flag.NewFlagSet("info", flag.ExitOnError) addGlobalFlags(flags) if err := flags.Parse(args); err != nil { return err } return printPackageInfo(str2slice(packageIds)...) } func enableBF(pkgId ...int) error { if len(pkgId) == 0 { fmt.Printf("Enabling BF for all packages\n") } else { fmt.Printf("Enabling BF for package(s) %v\n", pkgId) } err := sst.EnableBF(pkgId...) if err != nil { return err } return printPackageInfo(pkgId...) } func disableBF(pkgId ...int) error { if len(pkgId) == 0 { fmt.Printf("Disabling BF for all packages\n") } else { fmt.Printf("Disabling BF for package(s) %v\n", pkgId) } err := sst.DisableBF(pkgId...) if err != nil { return err } return printPackageInfo(pkgId...) } func subCmdBF(args []string) error { var enable, disable bool flags := flag.NewFlagSet("bf", flag.ExitOnError) flags.BoolVar(&enable, "enable", false, "enable feature") flags.BoolVar(&disable, "disable", false, "disable feature") addGlobalFlags(flags) if err := flags.Parse(args); err != nil { return err } if (!enable && !disable) || (enable && disable) { fmt.Printf("Please provide either -enable or -disable flag\n") return nil } var err error pkgs := str2slice(packageIds) if enable { err = enableBF(pkgs...) } else { err = disableBF(pkgs...) } return err } func getPackage(packageStr string, cpus utils.IDSet) (map[int]*sst.SstPackageInfo, *sst.SstPackageInfo, []int, error) { var infomap map[int]*sst.SstPackageInfo var info *sst.SstPackageInfo var packageId int var err error // If user has specified a package, then all the CPUs must belong to it. pkgs := str2slice(packageStr) if len(pkgs) > 1 { fmt.Printf("Only one package can be configured at a time (you have %d)\n", len(pkgs)) return nil, nil, nil, fmt.Errorf("Provide one package value only") } if len(pkgs) == 0 { // User has not specified a package, figure it out from the // first CPU in the list. infomap, err = sst.GetPackageInfo() if err != nil { return nil, nil, nil, err } for packageId, info = range infomap { if sst.CheckPackageCpus(info, cpus) { pkgs = append(pkgs, packageId) break } } } else { // User has specified one package, make sure all the CPUs belong to it. infomap, err = sst.GetPackageInfo(pkgs...) if err != nil { return nil, nil, nil, err } for packageId, info = range infomap { if !sst.CheckPackageCpus(info, cpus) { fmt.Printf("All the CPUs %v must belong to one specific package\n", cpus) return nil, nil, nil, fmt.Errorf("Not all CPUs belong to package %d", packageId) } pkgs = append(pkgs, packageId) break } } return infomap, info, pkgs, nil } // TODO: Instead of all CP parameters groupped together, separate them like this. // sst-ctl cp enable // sst-ctl cp disable // sst-ctl cp configure -clos=CLOS... // sst-ctl cp assign... func subCmdCP(args []string) error { var enable, disable, reset bool // Clos setup variables var epp, minFreq, maxFreq, desiredFreq, proportionalPriority, clos int // CPU to Clos mapping variables var cpuStr string var cpus utils.IDSet var packageId int var pkgs []int var info *sst.SstPackageInfo var infomap map[int]*sst.SstPackageInfo var err error var priority int defaultMaxFreq := 0xff flags := flag.NewFlagSet("cp", flag.ExitOnError) flags.BoolVar(&enable, "enable", false, "Enable feature") flags.BoolVar(&disable, "disable", false, "Disable feature") flags.BoolVar(&reset, "reset", false, "Reset CP to a known state") flags.IntVar(&clos, "clos", -1, "Class of service (0 - 3)") flags.IntVar(&epp, "epp", 0, "Energy Performance Preference value, Lower value favors performance, and higher value favors power. The value can be between 0 and 15. The default value is 0.") flags.IntVar(&minFreq, "min", 0, "Clos minimum frequency MHz") flags.IntVar(&maxFreq, "max", defaultMaxFreq, "Clos maximum frequency MHz") flags.IntVar(&desiredFreq, "desired", 0, "Clos desired frequency MHz") flags.IntVar(&proportionalPriority, "proportional", 0, "Clos proportional priority weight. Used if CP priority mode is 0 (Proportional)") flags.IntVar(&priority, "priority", 1, "CP priority mode. 0 is Proportional, 1 is Ordered.") flags.StringVar(&cpuStr, "cpus", "", "List of CPUs assigned to the Clos.") addGlobalFlags(flags) flags.Usage = func() { flags.PrintDefaults() fmt.Fprintf(os.Stderr, "\nExample usage:\n\n") fmt.Fprintf(os.Stderr, "First reset CP to default:\n\t%s cp -reset\n\n", os.Args[0]) fmt.Fprintf(os.Stderr, "Then set the CLOS values:\n\t%s cp -clos 1 -desired 280 -epp 1 -max 30 -min 21 -priority 1 -package 0\n\n", os.Args[0]) fmt.Fprintf(os.Stderr, "Then bind CPUs to a CLOS:\n\t%s cp -clos 1 -cpus 1,3,5,6\n\n", os.Args[0]) fmt.Fprintf(os.Stderr, "Finally enable CP:\n\t%s cp -enable -package 0\n\n", os.Args[0]) } if err := flags.Parse(args); err != nil { return err } if reset { err := sst.ResetCPConfig() _ = printPackageInfo() return err } if enable && disable { return fmt.Errorf("Please provide either -enable or -disable flag") } // If user specifies a list of CPUs, then he/she wants to assign those // CPUs to a specific CLOS. If the -cpus option is not set, then user // wants to configure the actual CLOS values. Both operations cannot be // done at the same time. // If user specifies a list of CPUs, then the package option is ignored. // Verify that all the CPUs belong to one specific package. if cpuStr != "" { cpus = utils.NewIDSet(str2slice(cpuStr)...) infomap, info, pkgs, err = getPackage(packageIds, cpus) if err != nil { return fmt.Errorf("Cannot get CPUs %v package: %w", cpus, err) } if len(pkgs) == 0 { return fmt.Errorf("All the CPUs %v must belong to one specific package", cpus) } if clos < 0 { return fmt.Errorf("Clos not set, use -clos option") } cpu2Clos := make(sst.ClosCPUSet, 1) cpu2Clos[clos] = cpus if err := sst.ConfigureCP(info, priority, &cpu2Clos); err != nil { return err } } else if clos >= 0 { pkgs = str2slice(packageIds) if len(pkgs) == 0 { return fmt.Errorf("No packages set, invalid value %q", packageIds) } closinfo := sst.SstClosInfo{ EPP: epp, ProportionalPriority: proportionalPriority, MinFreq: minFreq, MaxFreq: maxFreq, DesiredFreq: desiredFreq, } infomap, err = sst.GetPackageInfo(pkgs...) if err != nil { return fmt.Errorf("Cannot get package info: %w", err) } for _, info = range infomap { if err := sst.ClosSetup(info, clos, &closinfo); err != nil { return fmt.Errorf("Cannot set Clos: %w", err) } } } else { if (!enable && !disable) && clos < 0 { return fmt.Errorf("Clos not set, use -clos option") } // Print information if user just wants to enable / disable CP infomap, _ = sst.GetPackageInfo(pkgs...) } if enable || disable { for packageId, info = range infomap { if enable { fmt.Printf("Enabling CP for package %d\n", packageId) err = sst.EnableCP(info) if err != nil { return err } } else if disable { fmt.Printf("Disabling CP for package %d\n", packageId) err = sst.DisableCP(info) if err != nil { return err } } } } for packageId = range infomap { // If we add a CPU to Clos, punit might add another CPU to same Clos. // Make sure we have re-read the package info before printing it _ = printPackageInfo(packageId) } return nil } golang-github-intel-goresctrl-0.3.0/doc/000077500000000000000000000000001432612301500201165ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/doc/blockio.md000066400000000000000000000050531432612301500220650ustar00rootroot00000000000000# Block I/O ## Background The cgroup block I/O controller, [blkio](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/blkio-controller.html) in cgroup v1, [io](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html#io) in cgroup v2, in Linux kernel controls I/O scheduler weights and I/O bandwidth per block device. The blockio package in goresctrl is configured with class-based block I/O controller parameters, where different parameters can be configured for each class. The package provides two separate output options: parameters of a class can be applied directly to cgroup v1 directory structure, or they can be exported as [Linux BlockIO OCI spec](https://github.com/opencontainers/runtime-spec/blob/master/config-linux.md#block-io). ## API The API is described in [pkg.go.dev](https://pkg.go.dev/github.com/intel/goresctrl/pkg/blockio). ## Configuration Block I/O classes can be configured with a yaml file. Example: ``` Classes: # Define a blockio class "LowPrioThrottled". # Containers in this class will be throttled and handled as # low priority in the I/O scheduler. LowPrioThrottled: # Weight without a Devices list specifies the default # I/O scheduler weight for all devices # that are not explicitly mentioned in following items. # This will be written to cgroups(.bfq).weight. # Weights range from 10 to 1000, the default is 100. - Weight: 80 # Set all parameters for all /dev/sd* and /dev/vd* block # devices. - Devices: - /dev/sd[a-z] - /dev/vd[a-z] ThrottleReadBps: 50M # max read bytes per second ThrottleWriteBps: 10M # max write bytes per second ThrottleReadIOPS: 10k # max read io operations per second ThrottleWriteIOPS: 5k # max write io operations per second Weight: 50 # I/O scheduler (cfq/bfq) weight for # these devices will be written to # cgroups(.bfq).weight_device # Set parameters particularly for SSD devices. # This configuration overrides above configurations for those # /dev/sd* and /dev/vd* devices whose disk id contains "SSD". - Devices: - /dev/disk/by-id/*SSD* ThrottleReadBps: 100M ThrottleWriteBps: 40M # Not mentioning Throttle*IOPS means no I/O operations # throttling on matching devices. Weight: 50 # Define a blockio class "HighPrioFullSpeed". # There is no throttling on these containers, and # they will be prioritized by the I/O scheduler. HighPrioFullSpeed: - Weight: 400 ``` golang-github-intel-goresctrl-0.3.0/doc/rdt.md000066400000000000000000000261051432612301500212350ustar00rootroot00000000000000# Intel RDT (Resource Director Technology) ## Background Intel® RDT provides capabilities for cache and memory allocation and monitoring. In Linux system the functionality is exposed to the user space via the [resctrl](https://www.kernel.org/doc/Documentation/x86/intel_rdt_ui.txt) filesystem. Cache and memory allocation in RDT is handled by using resource control groups or classes of service (CLOSes). Resource allocation is specified on the group level and each task (process/thread) is assigned to one group. In the context of goresctrl the term 'RDT class' is used instead of 'resource control group' or 'CLOS'. Goresctrl supports all available RDT technologies, i.e. L2 and L3 Cache Allocation (CAT) with Code and Data Prioritization (CDP) and Memory Bandwidth Allocation (MBA) plus Cache Monitoring (CMT) and Memory Bandwidth Monitoring (MBM). ## API The API is described in [pkg.go.dev](https://pkg.go.dev/github.com/intel/goresctrl/pkg/rdt). # Configuration ## RDT Classes Goresctrl provides hiearachical approach for managing RDT resources. The RDT configuration is a two-level hierarchy consisting of partitions and classes: a set of partitions each having a set of classes. ### Partitions A partition consists of available resources and classes that share the resources. Resources include portions of caches (L2 and L3) and memory bandwidth (MB). Cache partitioning is exclusive: cache portions of two partitions are not allowed to overlap. However, by design of the underlying technology, MB allocations are not exclusive. Thus, it is possible to assign all partitions 100% of memory bandwidth, for example. ### Classes Classes represent the actual RDT classes processes are assigned to. In contrast to partitions, cache allocation between classes under a specific partition may overlap (and they usually do). Requirements for class specifications: - Names of classes must be unique accross all partitions - Total number of classes (CLOSes) supported by the underlying hardware must not be exceeded. - **NOTE:** resctrl root and possible groups managed outside goresctrl are also accounted against this limit. - Reserved name `DEFAULT` or an empty string refer to the resctrl root ## Configuration format ```yaml # Common options options: l2: # Set to false if L2 CAT must be available (Default is true). optional: [true|false] l3: # Set to false if L3 CAT must be available (Default is true). optional: [true|false] mb: # Set to false if MBA must be available (Default is true). optional: [true|false] partitions: : # L2 CAT configuration of the partition l2Allocation: : # L2 allocation spec used when CDP is not enabled, or, if CDP is # enabled but separate code and data specs are not specified unified: # L2 allocation spec for the code path when CDP is enabled (optional) code: # L2 allocation spec for the data path when CDP is enabled (optional) data: # L3 CAT configuration of the partition l3Allocation: : # L3 allocation spec used when CDP is not enabled, or, if CDP is # enabled but separate code and data specs are not specified unified: # L3 allocation spec for the code path when CDP is enabled (optional) code: # L3 allocation spec for the data path when CDP is enabled (optional) data: # MBA configuration of the partition mbAllocation: # MB allocation spec : classes: : l2Allocation: : # L2 allocation spec used when CDP is not enabled, or, if CDP is # enabled but separate code and data specs are not specified unified: # L2 allocation spec for the code path when CDP is enabled (optional) code: # L2 allocation spec for the data path when CDP is enabled (optional) data: l3Allocation: : # L3 allocation spec used when CDP is not enabled, or, if CDP is # enabled but separate code and data specs are not specified unified: # L3 allocation spec for the code path when CDP is enabled (optional) code: # L3 allocation spec for the data path when CDP is enabled (optional) data: mbAllocation: # MB allocation spec of the class : # Settings for the Kubernetes helper functions. Have no effect on the resctrl # configuration and control interface. kubernetes: # Set to true to deny assigning to this class via container annotation denyContainerAnnotation: [true|false] # Set to true to deny assigning to this class via pod annotation denyPodAnnotation: [true|false] ``` | Field | Format | Example | Description | | ----- | ------ | ------- | ----------- | | `` | string | `exclusive` | Name of a higher level RDT partition. | `` | string | `guaranteed` | Name of an RDT class, mapping to a directory in the resctrl fs. Reserved name `DEFAULT` or an empty string can be used to refer to the root class. | `` | cpuset (string) | `0,2,4,8-11` | Set of cache ids. Special value 'all' denotes a default used for cache "all the reset". | `` | percentage (string) | `"60%"` | Cache allocation spec, may be specified as relative (percentage) or absolute (bitmask). An absolute bitmask must be contiguous.   | hex bitmask (string) | `"0xf0"` |   | bit numbers (string) | `"0-3"` | | `` | list of strings | `[50%, 1000MBps]` | Memory bandwidth allocation spec, separarate values for percentage and MBps based allocation. The *MBps* value is in effect when resctrl is mounted with `-o mba_MBps`. ## Short forms The configuration accepts shortforms in order to allow easier and more readable configuration of the common and simple use cases. 1. Separate unified/code/data specs can be omitted, when no separate CDP config is desired, i.e. ``` : ``` is equal to ``` : unified: ``` 1. `` may be omitted if no cache id specific configuration (and no CDP config for CAT) is desired, i.e. ``` l3Allocation: "60%" mbAllocation: ["50%"] ``` is equal to ``` l3Allocation: all: unified: "60%" mbAllocation: all: ["50%"] ``` ## Examples Below is a config snippet that would allocate (ca.) 60% of the L3 cache lines exclusively to the guaranteed class. The remaining 40% L3 is for burstable and besteffort, Besteffort getting only 50% of this. guaranteed class gets full memory bandwidth whereas the other classes are throttled to 50%. ```yaml options: l2: optional: true l3: optional: true mb: optional: true partitions: exclusive: # Allocate 80% of all L2 cache IDs to the "exclusive" partition l2Allocation: "80%" # Allocate 60% of all L3 cache IDs to the "exclusive" partition l3Allocation: "60%" mbAllocation: ["100%"] classes: guaranteed: # Allocate all of the partitions cache lines and memory bandwidth to "guaranteed" l2Allocation: "100%" l3Allocation: "100%" # The class will get 100% by default #mbAllocation: ["100%"] shared: # Allocate 20% of L2 and 40% L3 cache IDs to the "shared" partition # These will NOT overlap with the cache lines allocated for "exclusive" partition l2Allocation: "20%" l3Allocation: "40%" mbAllocation: ["50%"] classes: burstable: # Allow "burstable" to use all cache lines of the "shared" partition l2Allocation: "100%" l3Allocation: "100%" # The class will get 100% by default #mbAllocation: ["100%"] besteffort: # Allow "besteffort" to use all L2 but only half of the L3 cache # lines of the "shared" partition. # These will overlap with those used by "burstable" l2Allocation: "100%" l3Allocation: "50%" # The class will get 100% by default #mbAllocation: ["100%"] DEFAULT: # Also configure the resctrl root that all processes in the system are # placed in by default l2Allocation: "50%" l3Allocation: "30%" # The class will get 100% by default #mbAllocation: ["100%"] ``` The configuration also supports far more fine-grained control, e.g. per cache-ID configuration (i.e. different cache ids, or sockets, having different allocation) and Code and Data Prioritization (CDP) allowing different cache allocation for code and data paths. ```yaml ... partitions: exclusive: l3Allocation: "60%" mbAllocation: ["100%"] classes: # Automatically gets 100% of what was allocated for the partition guaranteed: shared: l3Allocation: # 'all' denotes the default and must be specified all: "40%" # Specific cache allocation for cache-ids 2 and 3 2-3: "20%" mbAllocation: ["100%"] classes: burstable: l3Allocation: all: unified: "100%" code: "100%" data: "80%" mbAllocation: all: ["80%"] 2-3: ["50%"] ... ... ``` In addition, if the hardware details are known, raw bitmasks or bit numbers (`0x1f` or '0-4`) can be used instead of percentages in order to be able to configure cache allocations exactly as required. The bits in this case correspond to those in /sys/fs/resctrl/ bitmasks. You can also mix relative (percentage) and absolute (bitmask) allocations. For cases where the resctrl filesystem is mounted with `-o mba_MBps` Memory bandwidth must be specifed in MBps. ```yaml ... partitions: exclusive: # Specify bitmask in bit numbers l3Allocation: "8-19" # MBps value takes effect when resctrl mount option mba_MBps is used mbAllocation: ["100%", "100000MBps"] classes: # Automatically gets 100% of what was allocated for the partition guaranteed: shared: # Explicit bitmask l3Allocation: "0xff" mbAllocation: ["50%", "2000MBps"] classes: # burstable gets 100% of what was allocated for the partition burstable: besteffort: l3Allocation: "50%" # besteffort gets 50% of the 50% (i.e. 25% of total) or 1000MBps mbAllocation: ["50%", "1000MBps"] ``` ## Dynamic Configuration RDT supports dynamic configuration i.e. the parameters of existing classes may changed on-the-fly. golang-github-intel-goresctrl-0.3.0/doc/sst.md000066400000000000000000000003371432612301500212540ustar00rootroot00000000000000# Intel SST (Speed Select Technology) ## Background Intel® SST provides capabilities for managing CPU core frequency. ## API The API is described in [pkg.go.dev](https://pkg.go.dev/github.com/intel/goresctrl/pkg/sst). golang-github-intel-goresctrl-0.3.0/go.mod000066400000000000000000000020111432612301500204510ustar00rootroot00000000000000module github.com/intel/goresctrl go 1.18 require ( github.com/google/go-cmp v0.5.9 github.com/hashicorp/go-multierror v1.1.1 github.com/opencontainers/runtime-spec v1.0.2 github.com/prometheus/client_golang v1.13.0 golang.org/x/sys v0.0.0-20220908164124-27713097b956 k8s.io/apimachinery v0.25.0 sigs.k8s.io/yaml v1.3.0 ) require ( github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.1.2 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/protobuf v1.5.2 // indirect github.com/google/gofuzz v1.2.0 // indirect github.com/hashicorp/errwrap v1.1.0 // indirect github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect github.com/prometheus/client_model v0.2.0 // indirect github.com/prometheus/common v0.37.0 // indirect github.com/prometheus/procfs v0.8.0 // indirect google.golang.org/protobuf v1.28.1 // indirect gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect ) golang-github-intel-goresctrl-0.3.0/go.sum000066400000000000000000001427401432612301500205140ustar00rootroot00000000000000cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk= cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= github.com/go-kit/log v0.2.0/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0= github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo= github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/opencontainers/runtime-spec v1.0.2 h1:UfAcuLBJB9Coz72x1hgl8O5RVzTdNiaglX6v2DM6FI0= github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= github.com/prometheus/client_golang v1.13.0 h1:b71QUfeo5M8gq2+evJdTPfZhYMAU0uKPkyPJ7TPsloU= github.com/prometheus/client_golang v1.13.0/go.mod h1:vTeo+zgvILHsnnj/39Ou/1fPN5nJFOEMgftOUOmlvYQ= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/common v0.37.0 h1:ccBbHCgIiT9uSoFY0vX8H3zsNR5eLt17/RQLUvn8pXE= github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= github.com/prometheus/procfs v0.8.0 h1:ODq8ZFEaYeCaZOJlZZdJA2AbQR98dSHSM1KW/You5mo= github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220908164124-27713097b956 h1:XeJjHH1KiLpKGb6lvMiksZ9l0fVUh+AmGcm0nOMEBOY= golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA= google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY= google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60= google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk= google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w= google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= k8s.io/apimachinery v0.25.0 h1:MlP0r6+3XbkUG2itd6vp3oxbtdQLQI94fD5gCS+gnoU= k8s.io/apimachinery v0.25.0/go.mod h1:qMx9eAk0sZQGsXGu86fab8tZdffHbwUfsvzqKn4mfB0= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= golang-github-intel-goresctrl-0.3.0/pkg/000077500000000000000000000000001432612301500201325ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/blockio/000077500000000000000000000000001432612301500215545ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/blockio/blockio.go000066400000000000000000000411571432612301500235350ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // Package blockio implements class-based cgroup blockio controller // management for containers. // // Input: configuration of classes with blockio controller parameters // (weights, throttling) for sets of block devices. // // Outputs: // Option 1: Write blockio parameters of a class to a cgroup directory. // Option 2: Return blockio parameters of a class in a OCI LinuxBlockIO // structure, that can be passed to OCI-compliant container // runtime. // // Notes: // - Using Weight requires bfq or cfq I/O scheduler to be // effective for the block devices where Weight is used. // // Configuration example: // // Classes: // // # Define a blockio class "LowPrioThrottled". // # Containers in this class will be throttled and handled as // # low priority in the I/O scheduler. // // LowPrioThrottled: // // # Weight without a Devices list specifies the default // # I/O scheduler weight for all devices // # that are not explicitly mentioned in following items. // # This will be written to cgroups(.bfq).weight. // # Weights range from 10 to 1000, the default is 100. // // - Weight: 80 // // # Set all parameters for all /dev/sd* and /dev/vd* block // # devices. // // - Devices: // - /dev/sd[a-z] // - /dev/vd[a-z] // ThrottleReadBps: 50M # max read bytes per second // ThrottleWriteBps: 10M # max write bytes per second // ThrottleReadIOPS: 10k # max read io operations per second // ThrottleWriteIOPS: 5k # max write io operations per second // Weight: 50 # I/O scheduler (cfq/bfq) weight for // # these devices will be written to // # cgroups(.bfq).weight_device // // # Set parameters particularly for SSD devices. // # This configuration overrides above configurations for those // # /dev/sd* and /dev/vd* devices whose disk id contains "SSD". // // - Devices: // - /dev/disk/by-id/*SSD* // ThrottleReadBps: 100M // ThrottleWriteBps: 40M // # Not mentioning Throttle*IOPS means no I/O operations // # throttling on matching devices. // Weight: 50 // // # Define a blockio class "HighPrioFullSpeed". // # There is no throttling on these containers, and // # they will be prioritized by the I/O scheduler. // // HighPrioFullSpeed: // - Weight: 400 // // Usage example: // blockio.SetLogger(logrus.New()) // if err := blockio.SetConfigFromFile("/etc/containers/blockio.yaml", false); err != nil { // return err // } // // Output option 1: write directly to cgroup "/mytestgroup" // if err := blockio.SetCgroupClass("/mytestgroup", "LowPrioThrottled"); err != nil { // return err // } // // Output option 2: OCI LinuxBlockIO of a blockio class // if lbio, err := blockio.OciLinuxBlockIO("LowPrioThrottled"); err != nil { // return err // } else { // fmt.Printf("OCI LinuxBlockIO for LowPrioThrottled:\n%+v\n", lbio) // } package blockio import ( "fmt" "io/ioutil" stdlog "log" "os" "path/filepath" "sort" "strings" "syscall" "golang.org/x/sys/unix" "k8s.io/apimachinery/pkg/api/resource" "sigs.k8s.io/yaml" "github.com/hashicorp/go-multierror" "github.com/intel/goresctrl/pkg/cgroups" grclog "github.com/intel/goresctrl/pkg/log" ) const ( // sysfsBlockDeviceIOSchedulerPaths expands (with glob) to block device scheduler files. // If modified, check how to parse device node from expanded paths. sysfsBlockDeviceIOSchedulerPaths = "/sys/block/*/queue/scheduler" ) // tBlockDeviceInfo holds information on a block device to be configured. // As users can specify block devices using wildcards ("/dev/disk/by-id/*SSD*") // tBlockDeviceInfo.Origin is maintained for traceability: why this // block device is included in configuration. // tBlockDeviceInfo.DevNode contains resolved device node, like "/dev/sda". type tBlockDeviceInfo struct { Major int64 Minor int64 DevNode string Origin string } // Our logger instance. var log grclog.Logger = grclog.NewLoggerWrapper(stdlog.New(os.Stderr, "[ blockio ] ", 0)) // classBlockIO connects user-defined block I/O classes to // corresponding cgroups blockio controller parameters. var classBlockIO = map[string]cgroups.BlockIOParameters{} // SetLogger sets the logger instance to be used by the package. // Examples: // // Log to standard logger: // stdlog := log.New(os.Stderr, "blockio:", 0) // blockio.SetLogger(goresctrllog.NewLoggerWrapper(stdlog)) // // Log to logrus: // blockio.SetLogger(logrus.New()) func SetLogger(l grclog.Logger) { log = l } // SetConfigFromFile reads and applies blockio configuration from the // filesystem. func SetConfigFromFile(filename string, force bool) error { if data, err := ioutil.ReadFile(filename); err == nil { if err = SetConfigFromData(data, force); err != nil { return fmt.Errorf("failed to set configuration from file %q: %s", filename, err) } return nil } else { return fmt.Errorf("failed to read config file %q: %v", filename, err) } } // SetConfigFromData parses and applies configuration from data. func SetConfigFromData(data []byte, force bool) error { config := &Config{} if err := yaml.Unmarshal(data, &config); err != nil { return err } return SetConfig(config, force) } // SetConfig scans available block devices and applies new configuration. func SetConfig(opt *Config, force bool) error { if opt == nil { // Setting nil configuration clears current configuration. // SetConfigFromData([]byte(""), dontcare) arrives here. classBlockIO = map[string]cgroups.BlockIOParameters{} return nil } currentIOSchedulers, ioSchedulerDetectionError := getCurrentIOSchedulers() if ioSchedulerDetectionError != nil { log.Warnf("configuration validation partly disabled due to I/O scheduler detection error %#v", ioSchedulerDetectionError.Error()) } classBlockIO = map[string]cgroups.BlockIOParameters{} // Create cgroup blockio parameters for each blockio class for class := range opt.Classes { cgBlockIO, err := devicesParametersToCgBlockIO(opt.Classes[class], currentIOSchedulers) if err != nil { if force { log.Warnf("ignoring: %v", err) } else { return err } } classBlockIO[class] = cgBlockIO } return nil } // GetClasses returns block I/O class names func GetClasses() []string { classNames := make([]string, 0, len(classBlockIO)) for name := range classBlockIO { classNames = append(classNames, name) } sort.Strings(classNames) return classNames } // SetCgroupClass sets cgroup blkio controller parameters to match // blockio class. "group" is the cgroup directory of the container // without mountpoint and controller (blkio) directories: // "/kubepods/burstable/POD_ID/CONTAINER_ID". func SetCgroupClass(group string, class string) error { cgBlockIO, ok := classBlockIO[class] if !ok { return fmt.Errorf("no BlockIO parameters for class %#v", class) } err := cgroups.ResetBlkioParameters(group, cgBlockIO) if err != nil { return fmt.Errorf("assigning container in cgroup %q to class %#v failed: %w", group, class, err) } return nil } // getCurrentIOSchedulers returns currently active I/O scheduler used for each block device in the system. // Returns schedulers in a map: {"/dev/sda": "bfq"} func getCurrentIOSchedulers() (map[string]string, error) { var ios = map[string]string{} schedulerFiles, err := filepath.Glob(sysfsBlockDeviceIOSchedulerPaths) if err != nil { return ios, fmt.Errorf("error in I/O scheduler wildcards %#v: %w", sysfsBlockDeviceIOSchedulerPaths, err) } for _, schedulerFile := range schedulerFiles { devName := strings.SplitN(schedulerFile, "/", 5)[3] schedulerDataB, err := ioutil.ReadFile(schedulerFile) if err != nil { // A block device may be disconnected. log.Errorf("failed to read current I/O scheduler %#v: %v\n", schedulerFile, err) continue } schedulerData := strings.Trim(string(schedulerDataB), "\n") currentScheduler := "" if strings.IndexByte(schedulerData, ' ') == -1 { currentScheduler = schedulerData } else { openB := strings.Index(schedulerData, "[") closeB := strings.Index(schedulerData, "]") if -1 < openB && openB < closeB { currentScheduler = schedulerData[openB+1 : closeB] } } if currentScheduler == "" { log.Errorf("could not parse current scheduler in %#v\n", schedulerFile) continue } ios["/dev/"+devName] = currentScheduler } return ios, nil } // deviceParametersToCgBlockIO converts single blockio class parameters into cgroups blkio format. func devicesParametersToCgBlockIO(dps []DevicesParameters, currentIOSchedulers map[string]string) (cgroups.BlockIOParameters, error) { var errors *multierror.Error blkio := cgroups.NewBlockIOParameters() for _, dp := range dps { var err error var weight, throttleReadBps, throttleWriteBps, throttleReadIOPS, throttleWriteIOPS int64 weight, err = parseAndValidateQuantity("Weight", dp.Weight, -1, 10, 1000) errors = multierror.Append(errors, err) throttleReadBps, err = parseAndValidateQuantity("ThrottleReadBps", dp.ThrottleReadBps, -1, 0, -1) errors = multierror.Append(errors, err) throttleWriteBps, err = parseAndValidateQuantity("ThrottleWriteBps", dp.ThrottleWriteBps, -1, 0, -1) errors = multierror.Append(errors, err) throttleReadIOPS, err = parseAndValidateQuantity("ThrottleReadIOPS", dp.ThrottleReadIOPS, -1, 0, -1) errors = multierror.Append(errors, err) throttleWriteIOPS, err = parseAndValidateQuantity("ThrottleWriteIOPS", dp.ThrottleWriteIOPS, -1, 0, -1) errors = multierror.Append(errors, err) if dp.Devices == nil { if weight > -1 { blkio.Weight = weight } if throttleReadBps > -1 || throttleWriteBps > -1 || throttleReadIOPS > -1 || throttleWriteIOPS > -1 { errors = multierror.Append(errors, fmt.Errorf("ignoring throttling (rbps=%#v wbps=%#v riops=%#v wiops=%#v): Devices not listed", dp.ThrottleReadBps, dp.ThrottleWriteBps, dp.ThrottleReadIOPS, dp.ThrottleWriteIOPS)) } } else { blockDevices, err := currentPlatform.configurableBlockDevices(dp.Devices) if err != nil { // Problems in matching block device wildcards and resolving symlinks // are worth reporting, but must not block configuring blkio where possible. log.Warnf("%v", err) } if len(blockDevices) == 0 { log.Warnf("no matches on any of Devices: %v, parameters ignored", dp.Devices) } for _, blockDeviceInfo := range blockDevices { if weight != -1 { if ios, found := currentIOSchedulers[blockDeviceInfo.DevNode]; found { if ios != "bfq" && ios != "cfq" { log.Warnf("weight has no effect on device %#v due to "+ "incompatible I/O scheduler %#v (bfq or cfq required)", blockDeviceInfo.DevNode, ios) } } blkio.WeightDevice.Update(blockDeviceInfo.Major, blockDeviceInfo.Minor, weight) } if throttleReadBps != -1 { blkio.ThrottleReadBpsDevice.Update(blockDeviceInfo.Major, blockDeviceInfo.Minor, throttleReadBps) } if throttleWriteBps != -1 { blkio.ThrottleWriteBpsDevice.Update(blockDeviceInfo.Major, blockDeviceInfo.Minor, throttleWriteBps) } if throttleReadIOPS != -1 { blkio.ThrottleReadIOPSDevice.Update(blockDeviceInfo.Major, blockDeviceInfo.Minor, throttleReadIOPS) } if throttleWriteIOPS != -1 { blkio.ThrottleWriteIOPSDevice.Update(blockDeviceInfo.Major, blockDeviceInfo.Minor, throttleWriteIOPS) } } } } return blkio, errors.ErrorOrNil() } // parseAndValidateQuantity parses quantities, like "64 M", and validates that they are in given range. func parseAndValidateQuantity(fieldName string, fieldContent string, defaultValue int64, min int64, max int64) (int64, error) { // Returns field content if fieldContent == "" { return defaultValue, nil } qty, err := resource.ParseQuantity(fieldContent) if err != nil { return defaultValue, fmt.Errorf("syntax error in %#v (%#v)", fieldName, fieldContent) } value := qty.Value() if min != -1 && min > value { return defaultValue, fmt.Errorf("value of %#v (%#v) smaller than minimum (%#v)", fieldName, value, min) } if max != -1 && value > max { return defaultValue, fmt.Errorf("value of %#v (%#v) bigger than maximum (%#v)", fieldName, value, max) } return value, nil } // platformInterface includes functions that access the system. Enables mocking the system. type platformInterface interface { configurableBlockDevices(devWildcards []string) ([]tBlockDeviceInfo, error) } // defaultPlatform versions of platformInterface functions access the underlying system. type defaultPlatform struct{} // currentPlatform defines which platformInterface is used: defaultPlatform or a mock, for instance. var currentPlatform platformInterface = defaultPlatform{} // configurableBlockDevices finds major:minor numbers for device filenames. Wildcards are allowed in filenames. func (dpm defaultPlatform) configurableBlockDevices(devWildcards []string) ([]tBlockDeviceInfo, error) { // Return map {devNode: tBlockDeviceInfo} // Example: {"/dev/sda": {Major:8, Minor:0, Origin:"from symlink /dev/disk/by-id/ata-VendorXSSD from wildcard /dev/disk/by-id/*SSD*"}} var errors *multierror.Error blockDevices := []tBlockDeviceInfo{} var origin string // 1. Expand wildcards to device filenames (may be symlinks) // Example: devMatches["/dev/disk/by-id/ata-VendorSSD"] == "from wildcard \"dev/disk/by-id/*SSD*\"" devMatches := map[string]string{} // {devNodeOrSymlink: origin} for _, devWildcard := range devWildcards { devWildcardMatches, err := filepath.Glob(devWildcard) if err != nil { errors = multierror.Append(errors, fmt.Errorf("bad device wildcard %#v: %w", devWildcard, err)) continue } if len(devWildcardMatches) == 0 { errors = multierror.Append(errors, fmt.Errorf("device wildcard %#v does not match any device nodes", devWildcard)) continue } for _, devMatch := range devWildcardMatches { if devMatch != devWildcard { origin = fmt.Sprintf("from wildcard %#v", devWildcard) } else { origin = "" } devMatches[devMatch] = strings.TrimSpace(fmt.Sprintf("%v %v", devMatches[devMatch], origin)) } } // 2. Find out real device nodes behind symlinks // Example: devRealPaths["/dev/sda"] == "from symlink \"/dev/disk/by-id/ata-VendorSSD\"" devRealpaths := map[string]string{} // {devNode: origin} for devMatch, devOrigin := range devMatches { realDevNode, err := filepath.EvalSymlinks(devMatch) if err != nil { errors = multierror.Append(errors, fmt.Errorf("cannot filepath.EvalSymlinks(%#v): %w", devMatch, err)) continue } if realDevNode != devMatch { origin = fmt.Sprintf("from symlink %#v %v", devMatch, devOrigin) } else { origin = devOrigin } devRealpaths[realDevNode] = strings.TrimSpace(fmt.Sprintf("%v %v", devRealpaths[realDevNode], origin)) } // 3. Filter out everything but block devices that are not partitions // Example: blockDevices[0] == {Major: 8, Minor: 0, DevNode: "/dev/sda", Origin: "..."} for devRealpath, devOrigin := range devRealpaths { origin := "" if devOrigin != "" { origin = fmt.Sprintf(" (origin: %s)", devOrigin) } fileInfo, err := os.Stat(devRealpath) if err != nil { errors = multierror.Append(errors, fmt.Errorf("cannot os.Stat(%#v): %w%s", devRealpath, err, origin)) continue } fileMode := fileInfo.Mode() if fileMode&os.ModeDevice == 0 { errors = multierror.Append(errors, fmt.Errorf("file %#v is not a device%s", devRealpath, origin)) continue } if fileMode&os.ModeCharDevice != 0 { errors = multierror.Append(errors, fmt.Errorf("file %#v is a character device%s", devRealpath, origin)) continue } sys, ok := fileInfo.Sys().(*syscall.Stat_t) major := unix.Major(uint64(sys.Rdev)) minor := unix.Minor(uint64(sys.Rdev)) if !ok { errors = multierror.Append(errors, fmt.Errorf("cannot get syscall stat_t from %#v: %w%s", devRealpath, err, origin)) continue } if minor&0xf != 0 { errors = multierror.Append(errors, fmt.Errorf("skipping %#v: cannot weight/throttle partitions%s", devRealpath, origin)) continue } blockDevices = append(blockDevices, tBlockDeviceInfo{ Major: int64(major), Minor: int64(minor), DevNode: devRealpath, Origin: devOrigin, }) } return blockDevices, errors.ErrorOrNil() } golang-github-intel-goresctrl-0.3.0/pkg/blockio/blockio_test.go000066400000000000000000000301361432612301500245670ustar00rootroot00000000000000// Copyright 2019-2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package blockio import ( "fmt" "os" "path/filepath" "strings" "testing" "github.com/intel/goresctrl/pkg/cgroups" "github.com/intel/goresctrl/pkg/testutils" ) var knownIOSchedulers map[string]bool = map[string]bool{ "bfq": true, "cfq": true, "deadline": true, "kyber": true, "mq-deadline": true, "none": true, "noop": true, } // TestSetConfig: unit tests for SetConfigFromFile(), SetConfigFromData(), and SetConfig(). func TestSetConfig(t *testing.T) { initialConf := map[string]cgroups.BlockIOParameters{ "classname": cgroups.BlockIOParameters{}, } emptyConf := map[string]cgroups.BlockIOParameters{} goodConf := map[string]cgroups.BlockIOParameters{ "goodclass": cgroups.NewBlockIOParameters(), } classBlockIO = copyConf(initialConf) err := SetConfigFromFile("/blockio-test/non-existent-file", true) testutils.VerifyError(t, err, 1, []string{"/blockio-test/non-existent-file", "failed to read"}) testutils.VerifyDeepEqual(t, "effective configuration 1", initialConf, classBlockIO) badConfFile := testutils.CreateTempFile(t, "bad config contents.\n") emptyConfFile := testutils.CreateTempFile(t, "") goodConfFile := testutils.CreateTempFile(t, "Classes:\n goodclass:\n") defer os.Remove(badConfFile) defer os.Remove(emptyConfFile) defer os.Remove(goodConfFile) for syntaxerror := 0; syntaxerror < 4; syntaxerror++ { classBlockIO, err = copyConf(initialConf), nil switch syntaxerror { case 0: err = SetConfigFromFile(badConfFile, false) case 1: err = SetConfigFromFile(badConfFile, true) case 2: err = SetConfigFromData([]byte("bad config."), false) case 3: err = SetConfigFromData([]byte("bad config."), true) } if syntaxerror < 2 { testutils.VerifyError(t, err, 1, []string{badConfFile}) } testutils.VerifyError(t, err, 1, []string{"error unmarshaling"}) testutils.VerifyDeepEqual(t, fmt.Sprintf("syntax error configuration %d", syntaxerror), initialConf, classBlockIO) } // Test valid ways to clear (reset) all classes for clear := 0; clear < 8; clear++ { classBlockIO, err = copyConf(initialConf), nil switch clear { case 0: err = SetConfigFromFile(emptyConfFile, false) case 1: err = SetConfigFromFile(emptyConfFile, true) case 2: err = SetConfigFromData([]byte(""), false) case 3: err = SetConfigFromData([]byte(""), true) case 4: err = SetConfig(nil, false) case 5: err = SetConfig(nil, true) case 6: err = SetConfig(&Config{}, false) case 7: err = SetConfig(&Config{}, true) } testutils.VerifyNoError(t, err) testutils.VerifyDeepEqual(t, fmt.Sprintf("clear conf %d", clear), emptyConf, classBlockIO) } err = SetConfigFromFile(goodConfFile, true) testutils.VerifyNoError(t, err) testutils.VerifyDeepEqual(t, "ok conf", goodConf, classBlockIO) } // copyConf returns a shallow copy of blockio class configuration. func copyConf(orig map[string]cgroups.BlockIOParameters) map[string]cgroups.BlockIOParameters { result := map[string]cgroups.BlockIOParameters{} for key, value := range orig { result[key] = value } return result } func TestClassNames(t *testing.T) { classBlockIO = map[string]cgroups.BlockIOParameters{ "a": cgroups.BlockIOParameters{}, "z": cgroups.BlockIOParameters{}, "b": cgroups.BlockIOParameters{}, "x": cgroups.BlockIOParameters{}, "c": cgroups.BlockIOParameters{}, "d": cgroups.BlockIOParameters{}, } classes := GetClasses() testutils.VerifyStringSlices(t, []string{"a", "b", "c", "d", "x", "z"}, classes) classBlockIO = map[string]cgroups.BlockIOParameters{} classes = GetClasses() testutils.VerifyStringSlices(t, []string{}, classes) } // TestGetCurrentIOSchedulers: unit test for getCurrentIOSchedulers(). func TestGetCurrentIOSchedulers(t *testing.T) { currentIOSchedulers, err := getCurrentIOSchedulers() testutils.VerifyError(t, err, 0, nil) for blockDev, ioScheduler := range currentIOSchedulers { s, ok := knownIOSchedulers[ioScheduler] if !ok || !s { t.Errorf("unknown io scheduler %#v on block device %#v", ioScheduler, blockDev) } } } // TestConfigurableBlockDevices: unit tests for configurableBlockDevices(). func TestConfigurableBlockDevices(t *testing.T) { sysfsBlockDevs, err := filepath.Glob("/sys/block/*") if err != nil { sysfsBlockDevs = []string{} } devBlockDevs := []string{} for _, sysfsBlockDev := range sysfsBlockDevs { if strings.HasPrefix(sysfsBlockDev, "/sys/block/sd") || strings.HasPrefix(sysfsBlockDev, "/sys/block/vd") { devBlockDevs = append(devBlockDevs, strings.Replace(sysfsBlockDev, "/sys/block/", "/dev/", 1)) } } devPartitions := []string{} for _, devBlockDev := range devBlockDevs { devPartitions, _ = filepath.Glob(devBlockDev + "[0-9]") if len(devPartitions) > 0 { break } } t.Logf("test real block devices: %v", devBlockDevs) t.Logf("test partitions: %v", devPartitions) tcases := []struct { name string devWildcards []string expectedErrorCount int expectedErrorSubstrings []string expectedMatches int disabled bool disabledReason string }{ { name: "no device wildcards", devWildcards: nil, expectedErrorCount: 0, }, { name: "bad wildcard", devWildcards: []string{"/[-/verybadwildcard]"}, expectedErrorCount: 1, expectedErrorSubstrings: []string{"verybadwildcard", "syntax error"}, }, { name: "not matching wildcard", devWildcards: []string{"/dev/path that should not exist/*"}, expectedErrorCount: 1, expectedErrorSubstrings: []string{"does not match any"}, }, { name: "two wildcards: empty string and a character device", devWildcards: []string{"/dev/null", ""}, expectedErrorCount: 2, expectedErrorSubstrings: []string{"\"/dev/null\" is a character device", "\"\" does not match any"}, }, { name: "not a device or even a file", devWildcards: []string{"/proc", "/proc/meminfo", "/proc/notexistingfile"}, expectedErrorCount: 3, expectedErrorSubstrings: []string{"\"/proc\" is not a device", "\"/proc/meminfo\" is not a device"}, }, { name: "real block devices", devWildcards: devBlockDevs, expectedMatches: len(devBlockDevs), }, { name: "partition", devWildcards: devPartitions, expectedErrorCount: len(devPartitions), expectedErrorSubstrings: []string{"cannot weight/throttle partitions"}, disabled: len(devPartitions) == 0, disabledReason: "no block device partitions found", }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { if tc.disabled { t.Skip(tc.disabledReason) } realPlatform := defaultPlatform{} bdis, err := realPlatform.configurableBlockDevices(tc.devWildcards) testutils.VerifyError(t, err, tc.expectedErrorCount, tc.expectedErrorSubstrings) if len(bdis) != tc.expectedMatches { t.Errorf("expected %d matching block devices, got %d", tc.expectedMatches, len(bdis)) } }) } } // TestDevicesParametersToCgBlockIO: unit tests for devicesParametersToCgBlockIO(). func TestDevicesParametersToCgBlockIO(t *testing.T) { // switch real devicesParametersToCgBlockIO to call mockPlatform.configurableBlockDevices currentPlatform = mockPlatform{} tcases := []struct { name string dps []DevicesParameters iosched map[string]string expectedOci *cgroups.BlockIOParameters expectedErrorCount int expectedErrorSubstrings []string }{ { name: "all OCI fields", dps: []DevicesParameters{ { Weight: "144", }, { Devices: []string{"/dev/sda"}, ThrottleReadBps: "1G", ThrottleWriteBps: "2M", ThrottleReadIOPS: "3k", ThrottleWriteIOPS: "4", Weight: "50", }, }, iosched: map[string]string{"/dev/sda": "bfq"}, expectedOci: &cgroups.BlockIOParameters{ Weight: 144, WeightDevice: cgroups.DeviceWeights{ {Major: 11, Minor: 12, Weight: 50}, }, ThrottleReadBpsDevice: cgroups.DeviceRates{ {Major: 11, Minor: 12, Rate: 1000000000}, }, ThrottleWriteBpsDevice: cgroups.DeviceRates{ {Major: 11, Minor: 12, Rate: 2000000}, }, ThrottleReadIOPSDevice: cgroups.DeviceRates{ {Major: 11, Minor: 12, Rate: 3000}, }, ThrottleWriteIOPSDevice: cgroups.DeviceRates{ {Major: 11, Minor: 12, Rate: 4}, }, }, }, { name: "later match overrides value", dps: []DevicesParameters{ { Devices: []string{"/dev/sda", "/dev/sdb", "/dev/sdc"}, ThrottleReadBps: "100", Weight: "110", }, { Devices: []string{"/dev/sdb", "/dev/sdc"}, ThrottleReadBps: "300", Weight: "330", }, { Devices: []string{"/dev/sdb"}, ThrottleReadBps: "200", Weight: "220", }, }, iosched: map[string]string{"/dev/sda": "bfq", "/dev/sdb": "bfq", "/dev/sdc": "cfq"}, expectedOci: &cgroups.BlockIOParameters{ Weight: -1, WeightDevice: cgroups.DeviceWeights{ {Major: 11, Minor: 12, Weight: 110}, {Major: 21, Minor: 22, Weight: 220}, {Major: 31, Minor: 32, Weight: 330}, }, ThrottleReadBpsDevice: cgroups.DeviceRates{ {Major: 11, Minor: 12, Rate: 100}, {Major: 21, Minor: 22, Rate: 200}, {Major: 31, Minor: 32, Rate: 300}, }, }, }, { name: "invalid weights, many errors in different parameter sets", dps: []DevicesParameters{ { Weight: "99999", }, { Devices: []string{"/dev/sda"}, Weight: "1", }, { Devices: []string{"/dev/sdb"}, Weight: "-2", }, }, expectedErrorCount: 3, expectedErrorSubstrings: []string{ "(99999) bigger than maximum", "(1) smaller than minimum", "(-2) smaller than minimum", }, }, { name: "throttling without listing Devices", dps: []DevicesParameters{ { ThrottleReadBps: "100M", ThrottleWriteIOPS: "20k", }, }, expectedErrorCount: 1, expectedErrorSubstrings: []string{ "Devices not listed", "\"100M\"", "\"20k\"", }, }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { oci, err := devicesParametersToCgBlockIO(tc.dps, tc.iosched) testutils.VerifyError(t, err, tc.expectedErrorCount, tc.expectedErrorSubstrings) if tc.expectedOci != nil { testutils.VerifyDeepEqual(t, "OCI parameters", *tc.expectedOci, oci) } }) } } // mockPlatform implements mock versions of platformInterface functions. type mockPlatform struct{} // configurableBlockDevices mock always returns a set of block devices. func (mpf mockPlatform) configurableBlockDevices(devWildcards []string) ([]tBlockDeviceInfo, error) { blockDevices := []tBlockDeviceInfo{} for _, devWildcard := range devWildcards { if devWildcard == "/dev/sda" { blockDevices = append(blockDevices, tBlockDeviceInfo{ Major: 11, Minor: 12, DevNode: devWildcard, Origin: fmt.Sprintf("from wildcards %v", devWildcard), }) } else if devWildcard == "/dev/sdb" { blockDevices = append(blockDevices, tBlockDeviceInfo{ Major: 21, Minor: 22, DevNode: devWildcard, Origin: fmt.Sprintf("from wildcards %v", devWildcard), }) } else if devWildcard == "/dev/sdc" { blockDevices = append(blockDevices, tBlockDeviceInfo{ Major: 31, Minor: 32, DevNode: devWildcard, Origin: fmt.Sprintf("from wildcards %v", devWildcard), }) } } return blockDevices, nil } golang-github-intel-goresctrl-0.3.0/pkg/blockio/config.go000066400000000000000000000022361432612301500233530ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package blockio // Config contains a blockio configuration. type Config struct { // Classes define weights and throttling parameters for sets of devices. Classes map[string][]DevicesParameters `json:",omitempty"` } // DevicesParameters defines Block IO parameters for a set of devices. type DevicesParameters struct { Devices []string `json:",omitempty"` ThrottleReadBps string `json:",omitempty"` ThrottleWriteBps string `json:",omitempty"` ThrottleReadIOPS string `json:",omitempty"` ThrottleWriteIOPS string `json:",omitempty"` Weight string `json:",omitempty"` } golang-github-intel-goresctrl-0.3.0/pkg/blockio/kubernetes.go000066400000000000000000000035331432612301500242560ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package blockio import ( "github.com/intel/goresctrl/pkg/kubernetes" ) const ( // BlockioContainerAnnotation is the CRI level container annotation for setting // the blockio class of a container BlockioContainerAnnotation = "io.kubernetes.cri.blockio-class" // BlockioPodAnnotation is a Pod annotation for setting the blockio class of // all containers of the pod BlockioPodAnnotation = "blockio.resources.beta.kubernetes.io/pod" // BlockioPodAnnotationContainerPrefix is prefix for per-container Pod annotation // for setting the blockio class of one container of the pod BlockioPodAnnotationContainerPrefix = "blockio.resources.beta.kubernetes.io/container." ) // ContainerClassFromAnnotations determines the effective blockio // class of a container from the Pod annotations and CRI level // container annotations of a container. If the class is not specified // by any annotation, returns empty class name. Returned error is // reserved (nil). func ContainerClassFromAnnotations(containerName string, containerAnnotations, podAnnotations map[string]string) (string, error) { clsName, _ := kubernetes.ContainerClassFromAnnotations( BlockioContainerAnnotation, BlockioPodAnnotation, BlockioPodAnnotationContainerPrefix, containerName, containerAnnotations, podAnnotations) return clsName, nil } golang-github-intel-goresctrl-0.3.0/pkg/blockio/oci.go000066400000000000000000000042121432612301500226540ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package blockio import ( "fmt" oci "github.com/opencontainers/runtime-spec/specs-go" "github.com/intel/goresctrl/pkg/cgroups" ) // OciLinuxBlockIO returns OCI LinuxBlockIO structure corresponding to the class. func OciLinuxBlockIO(class string) (*oci.LinuxBlockIO, error) { blockio, ok := classBlockIO[class] if !ok { return nil, fmt.Errorf("no OCI BlockIO parameters for class %#v", class) } ociBlockio := oci.LinuxBlockIO{} if blockio.Weight != -1 { w := uint16(blockio.Weight) ociBlockio.Weight = &w } ociBlockio.WeightDevice = ociLinuxWeightDevices(blockio.WeightDevice) ociBlockio.ThrottleReadBpsDevice = ociLinuxThrottleDevices(blockio.ThrottleReadBpsDevice) ociBlockio.ThrottleWriteBpsDevice = ociLinuxThrottleDevices(blockio.ThrottleWriteBpsDevice) ociBlockio.ThrottleReadIOPSDevice = ociLinuxThrottleDevices(blockio.ThrottleReadIOPSDevice) ociBlockio.ThrottleWriteIOPSDevice = ociLinuxThrottleDevices(blockio.ThrottleWriteIOPSDevice) return &ociBlockio, nil } func ociLinuxWeightDevices(dws cgroups.DeviceWeights) []oci.LinuxWeightDevice { if len(dws) == 0 { return nil } olwds := make([]oci.LinuxWeightDevice, len(dws)) for i, wd := range dws { w := uint16(wd.Weight) olwds[i].Major = wd.Major olwds[i].Minor = wd.Minor olwds[i].Weight = &w } return olwds } func ociLinuxThrottleDevices(drs cgroups.DeviceRates) []oci.LinuxThrottleDevice { if len(drs) == 0 { return nil } oltds := make([]oci.LinuxThrottleDevice, len(drs)) for i, dr := range drs { oltds[i].Major = dr.Major oltds[i].Minor = dr.Minor oltds[i].Rate = uint64(dr.Rate) } return oltds } golang-github-intel-goresctrl-0.3.0/pkg/blockio/oci_test.go000066400000000000000000000116731432612301500237240ustar00rootroot00000000000000// Copyright 2019-2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package blockio import ( "testing" oci "github.com/opencontainers/runtime-spec/specs-go" "github.com/intel/goresctrl/pkg/cgroups" "github.com/intel/goresctrl/pkg/testutils" ) // TestOciLinuxBlockIO: unit tests for OciLinuxBlockIO(). func TestOciLinuxBlockIO(t *testing.T) { tcases := []struct { name string class string blockIOClasses map[string]cgroups.BlockIOParameters expectedBlockIO *oci.LinuxBlockIO // It would be great to define expected // oci.LinuxBlockIO with a single literal. But that // is impossible because Major and Minor fields are // inside non-exported oci.linuxBlockIODevice // struct. Therefore here are expected triplets of // major/minor/(weight|rate). expectedWeight uint16 expectedWeightDevices [][3]uint16 expectedThrottleReadBpsDevices [][3]uint64 expectedThrottleWriteBpsDevices [][3]uint64 expectedThrottleReadIOPSDevices [][3]uint64 expectedThrottleWriteIOPSDevices [][3]uint64 expectedErrorSubstrings []string }{ { name: "unknown class", class: "foobar", blockIOClasses: nil, expectedErrorSubstrings: []string{"foobar"}, }, { name: "all fields", class: "allfields", blockIOClasses: map[string]cgroups.BlockIOParameters{ "allfields": cgroups.BlockIOParameters{ Weight: 10, WeightDevice: cgroups.DeviceWeights{ {Major: 20, Minor: 21, Weight: 22}, {Major: 23, Minor: 24, Weight: 25}, }, ThrottleReadBpsDevice: cgroups.DeviceRates{ {Major: 30, Minor: 31, Rate: 32}, {Major: 33, Minor: 34, Rate: 35}, }, ThrottleWriteBpsDevice: cgroups.DeviceRates{ {Major: 40, Minor: 41, Rate: 42}, {Major: 43, Minor: 44, Rate: 45}, }, ThrottleReadIOPSDevice: cgroups.DeviceRates{ {Major: 50, Minor: 51, Rate: 52}, {Major: 53, Minor: 54, Rate: 55}, }, ThrottleWriteIOPSDevice: cgroups.DeviceRates{ {Major: 60, Minor: 61, Rate: 62}, {Major: 63, Minor: 64, Rate: 65}, }, }, }, expectedWeight: 10, expectedWeightDevices: [][3]uint16{{20, 21, 22}, {23, 24, 25}}, expectedThrottleReadBpsDevices: [][3]uint64{{30, 31, 32}, {33, 34, 35}}, expectedThrottleWriteBpsDevices: [][3]uint64{{40, 41, 42}, {43, 44, 45}}, expectedThrottleReadIOPSDevices: [][3]uint64{{50, 51, 52}, {53, 54, 55}}, expectedThrottleWriteIOPSDevices: [][3]uint64{{60, 61, 62}, {63, 64, 65}}, expectedBlockIO: &oci.LinuxBlockIO{}, }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { classBlockIO = tc.blockIOClasses gotBlockIO, gotError := OciLinuxBlockIO(tc.class) expectedErrorCount := 0 if len(tc.expectedErrorSubstrings) > 0 { expectedErrorCount = 1 } testutils.VerifyError(t, gotError, expectedErrorCount, tc.expectedErrorSubstrings) if tc.expectedBlockIO != nil { tc.expectedBlockIO.Weight = &tc.expectedWeight for _, wd := range tc.expectedWeightDevices { tc.expectedBlockIO.WeightDevice = append(tc.expectedBlockIO.WeightDevice, linuxWeightDevice(wd)) } for _, rd := range tc.expectedThrottleReadBpsDevices { tc.expectedBlockIO.ThrottleReadBpsDevice = append(tc.expectedBlockIO.ThrottleReadBpsDevice, linuxThrottleDevice(rd)) } for _, rd := range tc.expectedThrottleWriteBpsDevices { tc.expectedBlockIO.ThrottleWriteBpsDevice = append(tc.expectedBlockIO.ThrottleWriteBpsDevice, linuxThrottleDevice(rd)) } for _, rd := range tc.expectedThrottleReadIOPSDevices { tc.expectedBlockIO.ThrottleReadIOPSDevice = append(tc.expectedBlockIO.ThrottleReadIOPSDevice, linuxThrottleDevice(rd)) } for _, rd := range tc.expectedThrottleWriteIOPSDevices { tc.expectedBlockIO.ThrottleWriteIOPSDevice = append(tc.expectedBlockIO.ThrottleWriteIOPSDevice, linuxThrottleDevice(rd)) } } testutils.VerifyDeepEqual(t, "OCI BlockIO", tc.expectedBlockIO, gotBlockIO) }) } } func linuxWeightDevice(triplet [3]uint16) oci.LinuxWeightDevice { wd := oci.LinuxWeightDevice{} wd.Major = int64(triplet[0]) wd.Minor = int64(triplet[1]) wd.Weight = &triplet[2] return wd } func linuxThrottleDevice(triplet [3]uint64) oci.LinuxThrottleDevice { rd := oci.LinuxThrottleDevice{} rd.Major = int64(triplet[0]) rd.Minor = int64(triplet[1]) rd.Rate = triplet[2] return rd } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/000077500000000000000000000000001432612301500216145ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgroupblkio.go000066400000000000000000000262161432612301500244720ustar00rootroot00000000000000// Copyright 2020-2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package cgroups import ( "fmt" "strconv" "strings" "github.com/hashicorp/go-multierror" ) // cgroups blkio parameter filenames. var blkioWeightFiles = []string{"blkio.bfq.weight", "blkio.weight"} var blkioWeightDeviceFiles = []string{"blkio.bfq.weight_device", "blkio.weight_device"} var blkioThrottleReadBpsFiles = []string{"blkio.throttle.read_bps_device"} var blkioThrottleWriteBpsFiles = []string{"blkio.throttle.write_bps_device"} var blkioThrottleReadIOPSFiles = []string{"blkio.throttle.read_iops_device"} var blkioThrottleWriteIOPSFiles = []string{"blkio.throttle.write_iops_device"} // BlockIOParameters contains cgroups blockio controller parameters. // // Effects of Weight and Rate values in SetBlkioParameters(): // Value | Effect // -------+------------------------------------------------------------------- // -1 | Do not write to cgroups, value is missing. // 0 | Write to cgroups, will clear the setting as specified in cgroups blkio interface. // other | Write to cgroups, sets the value. type BlockIOParameters struct { Weight int64 WeightDevice DeviceWeights ThrottleReadBpsDevice DeviceRates ThrottleWriteBpsDevice DeviceRates ThrottleReadIOPSDevice DeviceRates ThrottleWriteIOPSDevice DeviceRates } // DeviceWeight contains values for // - blkio.[io-scheduler].weight type DeviceWeight struct { Major int64 Minor int64 Weight int64 } // DeviceRate contains values for // - blkio.throttle.read_bps_device // - blkio.throttle.write_bps_device // - blkio.throttle.read_iops_device // - blkio.throttle.write_iops_device type DeviceRate struct { Major int64 Minor int64 Rate int64 } // DeviceWeights contains weights for devices. type DeviceWeights []DeviceWeight // DeviceRates contains throttling rates for devices. type DeviceRates []DeviceRate // DeviceParameters interface provides functions common to DeviceWeights and DeviceRates. type DeviceParameters interface { Append(maj, min, val int64) Update(maj, min, val int64) } // Append appends (major, minor, value) to DeviceWeights slice. func (w *DeviceWeights) Append(maj, min, val int64) { *w = append(*w, DeviceWeight{Major: maj, Minor: min, Weight: val}) } // Append appends (major, minor, value) to DeviceRates slice. func (r *DeviceRates) Append(maj, min, val int64) { *r = append(*r, DeviceRate{Major: maj, Minor: min, Rate: val}) } // Update updates device weight in DeviceWeights slice, or appends it if not found. func (w *DeviceWeights) Update(maj, min, val int64) { for index, devWeight := range *w { if devWeight.Major == maj && devWeight.Minor == min { (*w)[index].Weight = val return } } w.Append(maj, min, val) } // Update updates device rate in DeviceRates slice, or appends it if not found. func (r *DeviceRates) Update(maj, min, val int64) { for index, devRate := range *r { if devRate.Major == maj && devRate.Minor == min { (*r)[index].Rate = val return } } r.Append(maj, min, val) } // NewBlockIOParameters creates new BlockIOParameters instance. func NewBlockIOParameters() BlockIOParameters { return BlockIOParameters{ Weight: -1, } } // NewDeviceWeight creates new DeviceWeight instance. func NewDeviceWeight() DeviceWeight { return DeviceWeight{ Major: -1, Minor: -1, Weight: -1, } } // NewDeviceRate creates new DeviceRate instance. func NewDeviceRate() DeviceRate { return DeviceRate{ Major: -1, Minor: -1, Rate: -1, } } type devMajMin struct { Major int64 Minor int64 } // ResetBlkioParameters adds new, changes existing and removes missing blockIO parameters in cgroupsDir. func ResetBlkioParameters(groupDir string, blockIO BlockIOParameters) error { var errors *multierror.Error oldBlockIO, _ := GetBlkioParameters(groupDir) newBlockIO := NewBlockIOParameters() newBlockIO.Weight = blockIO.Weight newBlockIO.WeightDevice = resetDevWeights(oldBlockIO.WeightDevice, blockIO.WeightDevice) newBlockIO.ThrottleReadBpsDevice = resetDevRates(oldBlockIO.ThrottleReadBpsDevice, blockIO.ThrottleReadBpsDevice) newBlockIO.ThrottleWriteBpsDevice = resetDevRates(oldBlockIO.ThrottleWriteBpsDevice, blockIO.ThrottleWriteBpsDevice) newBlockIO.ThrottleReadIOPSDevice = resetDevRates(oldBlockIO.ThrottleReadIOPSDevice, blockIO.ThrottleReadIOPSDevice) newBlockIO.ThrottleWriteIOPSDevice = resetDevRates(oldBlockIO.ThrottleWriteIOPSDevice, blockIO.ThrottleWriteIOPSDevice) errors = multierror.Append(errors, SetBlkioParameters(groupDir, newBlockIO)) return errors.ErrorOrNil() } // resetDevWeights adds wanted weight parameters to new and resets unwanted weights. func resetDevWeights(old, wanted []DeviceWeight) []DeviceWeight { new := []DeviceWeight{} seenDev := map[devMajMin]bool{} for _, wdp := range wanted { seenDev[devMajMin{wdp.Major, wdp.Minor}] = true new = append(new, wdp) } for _, wdp := range old { if !seenDev[devMajMin{wdp.Major, wdp.Minor}] { new = append(new, DeviceWeight{wdp.Major, wdp.Minor, 0}) } } return new } // resetDevRates adds wanted rate parameters to new and resets unwanted rates. func resetDevRates(old, wanted []DeviceRate) []DeviceRate { new := []DeviceRate{} seenDev := map[devMajMin]bool{} for _, rdp := range wanted { new = append(new, rdp) seenDev[devMajMin{rdp.Major, rdp.Minor}] = true } for _, rdp := range old { if !seenDev[devMajMin{rdp.Major, rdp.Minor}] { new = append(new, DeviceRate{rdp.Major, rdp.Minor, 0}) } } return new } // GetBlkioParameters returns BlockIO parameters from files in cgroups blkio controller directory. func GetBlkioParameters(group string) (BlockIOParameters, error) { var errors *multierror.Error blockIO := NewBlockIOParameters() errors = multierror.Append(errors, readWeight(group, blkioWeightFiles, &blockIO.Weight)) errors = multierror.Append(errors, readDeviceParameters(group, blkioWeightDeviceFiles, &blockIO.WeightDevice)) errors = multierror.Append(errors, readDeviceParameters(group, blkioThrottleReadBpsFiles, &blockIO.ThrottleReadBpsDevice)) errors = multierror.Append(errors, readDeviceParameters(group, blkioThrottleWriteBpsFiles, &blockIO.ThrottleWriteBpsDevice)) errors = multierror.Append(errors, readDeviceParameters(group, blkioThrottleReadIOPSFiles, &blockIO.ThrottleReadIOPSDevice)) errors = multierror.Append(errors, readDeviceParameters(group, blkioThrottleWriteIOPSFiles, &blockIO.ThrottleWriteIOPSDevice)) return blockIO, errors.ErrorOrNil() } // readWeight parses int64 from a cgroups entry. func readWeight(groupDir string, filenames []string, rv *int64) error { contents, err := readFirstFile(groupDir, filenames) if err != nil { return err } parsed, err := strconv.ParseInt(strings.TrimSuffix(contents, "\n"), 10, 64) if err != nil { return fmt.Errorf("parsing weight from %#v found in %v failed: %w", contents, filenames, err) } *rv = parsed return nil } // readDeviceParameters parses device lines used for weights and throttling rates. func readDeviceParameters(groupDir string, filenames []string, params DeviceParameters) error { var errors *multierror.Error contents, err := readFirstFile(groupDir, filenames) if err != nil { return err } for _, line := range strings.Split(contents, "\n") { // Device weight files may have "default NNN" line at the beginning. Skip it. if line == "" || strings.HasPrefix(line, "default ") { continue } // Expect syntax MAJOR:MINOR VALUE devVal := strings.Split(line, " ") if len(devVal) != 2 { errors = multierror.Append(errors, fmt.Errorf("invalid line %q, single space expected", line)) continue } majMin := strings.Split(devVal[0], ":") if len(majMin) != 2 { errors = multierror.Append(errors, fmt.Errorf("invalid line %q, single colon expected before space", line)) continue } major, majErr := strconv.ParseInt(majMin[0], 10, 64) minor, minErr := strconv.ParseInt(majMin[1], 10, 64) value, valErr := strconv.ParseInt(devVal[1], 10, 64) if majErr != nil || minErr != nil || valErr != nil { errors = multierror.Append(errors, fmt.Errorf("invalid number when parsing \"major:minor value\" from \"%s:%s %s\"", majMin[0], majMin[1], devVal[1])) continue } params.Append(major, minor, value) } return errors.ErrorOrNil() } // readFirstFile returns contents of the first successfully read entry. func readFirstFile(groupDir string, filenames []string) (string, error) { var errors *multierror.Error // If reading all the files fails, return list of read errors. for _, filename := range filenames { content, err := Blkio.Group(groupDir).Read(filename) if err == nil { return content, nil } errors = multierror.Append(errors, err) } err := errors.ErrorOrNil() if err != nil { return "", fmt.Errorf("could not read any of files %q: %w", filenames, err) } return "", nil } // SetBlkioParameters writes BlockIO parameters to files in cgroups blkio contoller directory. func SetBlkioParameters(group string, blockIO BlockIOParameters) error { var errors *multierror.Error if blockIO.Weight >= 0 { errors = multierror.Append(errors, writeFirstFile(group, blkioWeightFiles, "%d", blockIO.Weight)) } for _, wd := range blockIO.WeightDevice { errors = multierror.Append(errors, writeFirstFile(group, blkioWeightDeviceFiles, "%d:%d %d", wd.Major, wd.Minor, wd.Weight)) } for _, rd := range blockIO.ThrottleReadBpsDevice { errors = multierror.Append(errors, writeFirstFile(group, blkioThrottleReadBpsFiles, "%d:%d %d", rd.Major, rd.Minor, rd.Rate)) } for _, rd := range blockIO.ThrottleWriteBpsDevice { errors = multierror.Append(errors, writeFirstFile(group, blkioThrottleWriteBpsFiles, "%d:%d %d", rd.Major, rd.Minor, rd.Rate)) } for _, rd := range blockIO.ThrottleReadIOPSDevice { errors = multierror.Append(errors, writeFirstFile(group, blkioThrottleReadIOPSFiles, "%d:%d %d", rd.Major, rd.Minor, rd.Rate)) } for _, rd := range blockIO.ThrottleWriteIOPSDevice { errors = multierror.Append(errors, writeFirstFile(group, blkioThrottleWriteIOPSFiles, "%d:%d %d", rd.Major, rd.Minor, rd.Rate)) } return errors.ErrorOrNil() } // writeFirstFile writes content to the first existing file in the list under groupDir. func writeFirstFile(groupDir string, filenames []string, format string, args ...interface{}) error { var errors *multierror.Error // Returns list of errors from writes, list of single error due to all filenames missing or nil on success. for _, filename := range filenames { if err := Blkio.Group(groupDir).Write(filename, format, args...); err != nil { errors = multierror.Append(errors, err) continue } return nil } err := errors.ErrorOrNil() if err != nil { data := fmt.Sprintf(format, args...) return fmt.Errorf("writing all files %v failed, errors: %w, content %q", filenames, err, data) } return nil } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgroupblkio_test.go000066400000000000000000000501001432612301500255160ustar00rootroot00000000000000// Copyright 2020-2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package cgroups import ( "bytes" "io" "syscall" "testing" "github.com/intel/goresctrl/pkg/testutils" ) func TestUpdateAppend(t *testing.T) { tcases := []struct { name string inputMajMinVals [][]int64 inputItem []int64 expectedMajMinVal [][]int64 expectedErrorCount int expectedErrorSubstrings []string }{ { name: "update empty list", inputItem: []int64{1, 2, 3}, expectedMajMinVal: [][]int64{{1, 2, 3}}, }, { name: "update appends non-existing element", inputMajMinVals: [][]int64{{10, 20, 30}, {40, 50, 60}}, inputItem: []int64{1, 2, 3}, expectedMajMinVal: [][]int64{{10, 20, 30}, {40, 50, 60}, {1, 2, 3}}, }, { name: "update the first existing element", inputMajMinVals: [][]int64{{10, 20, 30}, {40, 50, 60}, {40, 50, 60}}, inputItem: []int64{40, 50, 66}, expectedMajMinVal: [][]int64{{10, 20, 30}, {40, 50, 66}, {40, 50, 60}}, }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { devWeights := DeviceWeights{} devRates := DeviceRates{} expDevWeights := DeviceWeights{} expDevRates := DeviceRates{} for _, item := range tc.inputMajMinVals { devWeights.Append(item[0], item[1], item[2]) devRates.Append(item[0], item[1], item[2]) } devWeights.Update(tc.inputItem[0], tc.inputItem[1], tc.inputItem[2]) devRates.Update(tc.inputItem[0], tc.inputItem[1], tc.inputItem[2]) for _, item := range tc.expectedMajMinVal { expDevWeights = append(expDevWeights, DeviceWeight{item[0], item[1], item[2]}) expDevRates = append(expDevRates, DeviceRate{item[0], item[1], item[2]}) } testutils.VerifyDeepEqual(t, "device weights", expDevWeights, devWeights) testutils.VerifyDeepEqual(t, "device rates", expDevRates, devRates) }) } } var fsBlkioUtFiles map[string]mockFile = map[string]mockFile{ mountDir + "/blkio/mockpods/clean/blkio.bfq.weight": {data: []byte("100\n")}, mountDir + "/blkio/mockpods/clean/blkio.bfq.weight_device": {}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_bps_device": {}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_bps_device": {}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_iops_device": {}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_iops_device": {}, mountDir + "/blkio/mockpods/no-blkio-bfq-weight/blkio.weight": {data: []byte("100\n")}, mountDir + "/blkio/mockpods/reset/blkio.bfq.weight": {data: []byte("200\n")}, mountDir + "/blkio/mockpods/reset/blkio.bfq.weight_device": {data: []byte("default 200\n1:2 3\n4:5 6\n")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.read_bps_device": {data: []byte("11:12 13\n14:15 16\n")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.write_bps_device": {data: []byte("21:22 23\n")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.read_iops_device": {data: []byte("31:32 33\n")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.write_iops_device": {data: []byte("41:42 43\n")}, mountDir + "/blkio/mockpods/merge/blkio.bfq.weight": {data: []byte("200\n")}, mountDir + "/blkio/mockpods/merge/blkio.bfq.weight_device": {data: []byte("default 200\n1:2 3\n4:5 6\n7:8 9")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.read_bps_device": {data: []byte("11:12 13\n14:15 16\n")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.write_bps_device": {data: []byte("21:22 23\n24:25 26\n")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.read_iops_device": {data: []byte("31:32 33\n331:332 333\n")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.write_iops_device": {data: []byte("41:42 43\n441:442 443\n")}, // parseok: // test weight without linefeed // test weight_device file with real "default" line // test parsing two lines and skipping empty lines // test single line file // test single line, missing LF at the end // test small and large values mountDir + "/blkio/parseok/blkio.bfq.weight": {data: []byte("1")}, mountDir + "/blkio/parseok/blkio.bfq.weight_device": {data: []byte("default 10\n1:2 3\n")}, mountDir + "/blkio/parseok/blkio.throttle.read_bps_device": {data: []byte("\n11:22 33\n\n111:222 333\n")}, mountDir + "/blkio/parseok/blkio.throttle.write_bps_device": {data: []byte("1111:2222 3333\n")}, mountDir + "/blkio/parseok/blkio.throttle.read_iops_device": {data: []byte("11111:22222 33333")}, mountDir + "/blkio/parseok/blkio.throttle.write_iops_device": {data: []byte("0:0 0\n4294967296:4294967297 9223372036854775807\n")}, // parse-err: // weight: not a number // weight_device: test bad line in the middle // read_bps_device: test no spaces // write_bps_device: test too many spaces // read_iobps_device: test no colons // write_iobps_device: test missing number mountDir + "/blkio/parse-err/blkio.bfq.weight": {data: []byte("xyz")}, mountDir + "/blkio/parse-err/blkio.bfq.weight_device": {data: []byte("default 10\n1:2 3\nbad\n4:5 6\n")}, mountDir + "/blkio/parse-err/blkio.throttle.read_bps_device": {data: []byte("11:22:33")}, mountDir + "/blkio/parse-err/blkio.throttle.write_bps_device": {data: []byte("1111 2222 3333 \n")}, mountDir + "/blkio/parse-err/blkio.throttle.read_iops_device": {data: []byte("1111122222 33333")}, mountDir + "/blkio/parse-err/blkio.throttle.write_iops_device": {data: []byte("0: 0\n")}, mountDir + "/blkio/write-enodev/blkio.bfq.weight": {write: func([]byte) (int, error) { return 0, syscall.ENODEV }}, mountDir + "/blkio/write-enodev/blkio.bfq.weight_device": {write: func([]byte) (int, error) { return 0, syscall.ENODEV }}, mountDir + "/blkio/write-enodev/blkio.throttle.read_bps_device": {write: func([]byte) (int, error) { return 0, syscall.ENODEV }}, mountDir + "/blkio/write-enodev/blkio.throttle.write_bps_device": {write: func([]byte) (int, error) { return 0, syscall.ENODEV }}, mountDir + "/blkio/write-enodev/blkio.throttle.read_iops_device": {write: func([]byte) (int, error) { return 0, syscall.ENODEV }}, mountDir + "/blkio/write-enodev/blkio.throttle.write_iops_device": {write: func([]byte) (int, error) { return 0, syscall.ENODEV }}, } // TestResetBlkioParameters: unit test for ResetBlkioParameters() func TestResetBlkioParameters(t *testing.T) { tcases := []struct { name string fsi fsiIface cntnrDir string blockIO BlockIOParameters expectedFsWrites map[string][][]byte expectedBlockIO *BlockIOParameters expectedErrorCount int expectedErrorSubstrings []string }{ { name: "write to clean cgroups", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "mockpods/clean", blockIO: BlockIOParameters{ Weight: 222, WeightDevice: DeviceWeights{{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}, ThrottleReadBpsDevice: DeviceRates{{11, 12, 13}, {111, 112, 113}}, ThrottleWriteBpsDevice: DeviceRates{{21, 22, 23}, {221, 222, 223}}, ThrottleReadIOPSDevice: DeviceRates{{31, 32, 33}, {331, 332, 333}}, ThrottleWriteIOPSDevice: DeviceRates{{41, 42, 43}, {441, 442, 443}}, }, expectedFsWrites: map[string][][]byte{ mountDir + "/blkio/mockpods/clean/blkio.bfq.weight": {[]byte("222")}, mountDir + "/blkio/mockpods/clean/blkio.bfq.weight_device": {[]byte("1:2 3"), []byte("4:5 6"), []byte("7:8 9")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_bps_device": {[]byte("11:12 13"), []byte("111:112 113")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_bps_device": {[]byte("21:22 23"), []byte("221:222 223")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_iops_device": {[]byte("31:32 33"), []byte("331:332 333")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_iops_device": {[]byte("41:42 43"), []byte("441:442 443")}, }, }, { name: "reset all existing", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "mockpods/reset", blockIO: NewBlockIOParameters(), expectedFsWrites: map[string][][]byte{ mountDir + "/blkio/mockpods/reset/blkio.bfq.weight_device": {[]byte("1:2 0"), []byte("4:5 0")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.read_bps_device": {[]byte("11:12 0"), []byte("14:15 0")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.write_bps_device": {[]byte("21:22 0")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.read_iops_device": {[]byte("31:32 0")}, mountDir + "/blkio/mockpods/reset/blkio.throttle.write_iops_device": {[]byte("41:42 0")}, }, }, { name: "merge", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "mockpods/merge", blockIO: BlockIOParameters{ Weight: 80, WeightDevice: DeviceWeights{{1, 2, 1113}, {7, 8, 9}}, // drop middle, update first, keep last ThrottleReadBpsDevice: DeviceRates{{11, 12, 13}}, // keep the first entry ThrottleWriteBpsDevice: DeviceRates{{24, 25, 26}}, // keep the last entry ThrottleReadIOPSDevice: DeviceRates{{31, 32, 33}, {331, 332, 333}}, // keep all ThrottleWriteIOPSDevice: DeviceRates{{41, 42, 430}, {441, 442, 4430}}, // change all }, expectedFsWrites: map[string][][]byte{ mountDir + "/blkio/mockpods/merge/blkio.bfq.weight": {[]byte("80")}, mountDir + "/blkio/mockpods/merge/blkio.bfq.weight_device": {[]byte("1:2 1113"), []byte("7:8 9"), []byte("4:5 0")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.read_bps_device": {[]byte("11:12 13"), []byte("14:15 0")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.write_bps_device": {[]byte("24:25 26"), []byte("21:22 0")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.read_iops_device": {[]byte("31:32 33"), []byte("331:332 333")}, mountDir + "/blkio/mockpods/merge/blkio.throttle.write_iops_device": {[]byte("41:42 430"), []byte("441:442 4430")}, }, }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { fsi = tc.fsi err := ResetBlkioParameters(tc.cntnrDir, tc.blockIO) testutils.VerifyError(t, err, tc.expectedErrorCount, tc.expectedErrorSubstrings) validateWriteHistory(t, tc.expectedFsWrites, fsi.(*fsMock).files) }) } } // validateWriteHistory compares expected writes to filesystem to all // observed writes. func validateWriteHistory(t *testing.T, expected map[string][][]byte, filesystem map[string]*mockFile) { for expFilename, expWrites := range expected { mf, ok := filesystem[expFilename] if !ok { t.Errorf("expected writes to %q, but file is missing", expFilename) return } obsWrites := mf.writeHistory if len(expWrites) != len(obsWrites) { t.Errorf("unexpected number of writes to %q: expected %v, observed %v", expFilename, expWrites, obsWrites) return } for i, expWrite := range expWrites { if !bytes.Equal(expWrite, obsWrites[i]) { t.Errorf("write at index %d differs: expected %v, observed %v", i, expWrites, obsWrites) } } } for obsFilename, mf := range filesystem { if mf.writeHistory != nil { if _, ok := expected[obsFilename]; !ok { t.Errorf("writes to unexpected file %q, observed: %v", obsFilename, mf.writeHistory) } } } } // TestGetBlkioParameters: unit test for GetBlkioParameters() func TestGetBlkioParameters(t *testing.T) { tcases := []struct { name string fsi fsiIface fsFuncs map[string]mockFile cntnrDir string readsFail int fsContent map[string]string expectedBlockIO *BlockIOParameters expectedErrorCount int expectedErrorSubstrings []string }{ { name: "all clean and empty", fsi: NewFsiMock(fsBlkioUtFiles), fsFuncs: map[string]mockFile{ // reuse clean directory, but force weight file empty mountDir + "/blkio/mockpods/clean/blkio.bfq.weight": { read: func([]byte) (int, error) { return 0, io.EOF }, }, }, cntnrDir: "mockpods/clean", expectedBlockIO: &BlockIOParameters{Weight: -1}, expectedErrorCount: 1, // weight is not expected to be empty expectedErrorSubstrings: []string{"parsing weight"}, }, { name: "everything defined", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/parseok", expectedBlockIO: &BlockIOParameters{ Weight: 1, WeightDevice: DeviceWeights{{1, 2, 3}}, ThrottleReadBpsDevice: DeviceRates{{11, 22, 33}, {111, 222, 333}}, ThrottleWriteBpsDevice: DeviceRates{{1111, 2222, 3333}}, ThrottleReadIOPSDevice: DeviceRates{{11111, 22222, 33333}}, ThrottleWriteIOPSDevice: DeviceRates{{0, 0, 0}, {4294967296, 4294967297, 9223372036854775807}}, }, }, { name: "test bad files", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/parse-err", expectedErrorCount: 6, expectedErrorSubstrings: []string{"bad", "xyz", "11:22:33", "1111 2222 3333 ", "1111122222 33333", "0: 0"}, expectedBlockIO: &BlockIOParameters{ Weight: -1, WeightDevice: DeviceWeights{{1, 2, 3}, {4, 5, 6}}, }, }, { name: "all files missing", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/this/container/does/not/exist", expectedBlockIO: &BlockIOParameters{Weight: -1}, expectedErrorCount: 6, expectedErrorSubstrings: []string{ "file not found", "blkio.bfq.weight", "blkio.bfq.weight_device", "blkio.throttle.read_bps_device", "blkio.throttle.write_bps_device", "blkio.throttle.read_iops_device", "blkio.throttle.write_iops_device", }, }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { fsi = tc.fsi overrideFsFuncs(fsi.(*fsMock), tc.fsFuncs) blockIO, err := GetBlkioParameters(tc.cntnrDir) testutils.VerifyError(t, err, tc.expectedErrorCount, tc.expectedErrorSubstrings) if tc.expectedBlockIO != nil { testutils.VerifyDeepEqual(t, "blockio parameters", *tc.expectedBlockIO, blockIO) } }) } } // overrideFsFuncs (re)sets user overrides of file-specific functions // in the mock filesystem. func overrideFsFuncs(fsm *fsMock, fsFuncs map[string]mockFile) { for filename, mf := range fsFuncs { if mf.open != nil { fsm.files[filename].open = mf.open } if mf.read != nil { fsm.files[filename].read = mf.read } if mf.write != nil { fsm.files[filename].write = mf.write } } } // TestSetBlkioParameters: unit test for SetBlkioParameters() func TestSetBlkioParameters(t *testing.T) { tcases := []struct { name string fsi fsiIface fsFuncs map[string]mockFile cntnrDir string blockIO BlockIOParameters writesFail int expectedFsWrites map[string][][]byte expectedErrorCount int expectedErrorSubstrings []string }{ { name: "write full OCI struct", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/mockpods/clean", blockIO: BlockIOParameters{ Weight: 10, WeightDevice: DeviceWeights{{Major: 1, Minor: 2, Weight: 3}}, ThrottleReadBpsDevice: DeviceRates{{Major: 11, Minor: 12, Rate: 13}}, ThrottleWriteBpsDevice: DeviceRates{{Major: 21, Minor: 22, Rate: 23}}, ThrottleReadIOPSDevice: DeviceRates{{Major: 31, Minor: 32, Rate: 33}}, ThrottleWriteIOPSDevice: DeviceRates{{Major: 41, Minor: 42, Rate: 43}}, }, expectedFsWrites: map[string][][]byte{ mountDir + "/blkio/mockpods/clean/blkio.bfq.weight": {[]byte("10")}, mountDir + "/blkio/mockpods/clean/blkio.bfq.weight_device": {[]byte("1:2 3")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_bps_device": {[]byte("11:12 13")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_bps_device": {[]byte("21:22 23")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_iops_device": {[]byte("31:32 33")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_iops_device": {[]byte("41:42 43")}, }, }, { name: "write empty struct", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/mockpods/clean", blockIO: BlockIOParameters{}, expectedFsWrites: map[string][][]byte{ mountDir + "/blkio/mockpods/clean/blkio.bfq.weight": {[]byte("0")}, }, }, { name: "multidevice weight and throttling, no weight write on -1", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/mockpods/clean", blockIO: BlockIOParameters{ Weight: -1, WeightDevice: DeviceWeights{{1, 2, 3}, {4, 5, 6}}, ThrottleReadBpsDevice: DeviceRates{{11, 12, 13}, {111, 112, 113}}, ThrottleWriteBpsDevice: DeviceRates{{21, 22, 23}, {221, 222, 223}}, ThrottleReadIOPSDevice: DeviceRates{{31, 32, 33}, {331, 332, 333}}, ThrottleWriteIOPSDevice: DeviceRates{{41, 42, 43}, {441, 442, 443}}, }, expectedFsWrites: map[string][][]byte{ mountDir + "/blkio/mockpods/clean/blkio.bfq.weight_device": {[]byte("1:2 3"), []byte("4:5 6")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_bps_device": {[]byte("11:12 13"), []byte("111:112 113")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_bps_device": {[]byte("21:22 23"), []byte("221:222 223")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.read_iops_device": {[]byte("31:32 33"), []byte("331:332 333")}, mountDir + "/blkio/mockpods/clean/blkio.throttle.write_iops_device": {[]byte("41:42 43"), []byte("441:442 443")}, }, }, { name: "no bfq.weight", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/mockpods/no-blkio-bfq-weight", blockIO: BlockIOParameters{Weight: 100}, writesFail: 1, expectedFsWrites: map[string][][]byte{ mountDir + "/blkio/mockpods/no-blkio-bfq-weight/blkio.weight": {[]byte("100")}, }, }, { name: "all writes fail", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "write-enodev", blockIO: BlockIOParameters{ Weight: -1, WeightDevice: DeviceWeights{{1, 0, 100}}, ThrottleReadBpsDevice: DeviceRates{{11, 12, 13}}, }, expectedErrorCount: 2, expectedErrorSubstrings: []string{ "\"1:0 100\"", "\"11:12 13\"", "\"blkio.bfq.weight_device\"", "\"blkio.weight_device\"", "read_bps_device", "no such device", "file not found", }, }, { name: "all files missing", fsi: NewFsiMock(fsBlkioUtFiles), cntnrDir: "/this/container/does/not/exist", blockIO: BlockIOParameters{ Weight: 10, WeightDevice: DeviceWeights{{Major: 1, Minor: 2, Weight: 3}}, ThrottleReadBpsDevice: DeviceRates{{Major: 11, Minor: 12, Rate: 13}}, ThrottleWriteBpsDevice: DeviceRates{{Major: 21, Minor: 22, Rate: 23}}, ThrottleReadIOPSDevice: DeviceRates{{Major: 31, Minor: 32, Rate: 33}}, ThrottleWriteIOPSDevice: DeviceRates{{Major: 41, Minor: 42, Rate: 43}}, }, expectedErrorCount: 6, expectedErrorSubstrings: []string{ "file not found", "blkio.bfq.weight", "blkio.bfq.weight_device", "blkio.throttle.read_bps_device", "blkio.throttle.write_bps_device", "blkio.throttle.read_iops_device", "blkio.throttle.write_iops_device", }, }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { fsi = tc.fsi overrideFsFuncs(fsi.(*fsMock), tc.fsFuncs) err := SetBlkioParameters(tc.cntnrDir, tc.blockIO) testutils.VerifyError(t, err, tc.expectedErrorCount, tc.expectedErrorSubstrings) validateWriteHistory(t, tc.expectedFsWrites, fsi.(*fsMock).files) }) } } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgroupcontrol.go000066400000000000000000000136341432612301500250520ustar00rootroot00000000000000// Copyright 2020-2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package cgroups import ( "bufio" "bytes" "errors" "fmt" "os" "path" "strings" "syscall" ) // Controller is our enumerated type for cgroup controllers. type Controller int // Group represents a control group. type Group string //nolint const ( // UnkownController represents a controller of unknown type. UnknownController Controller = iota // blkio cgroup controller. Blkio // cpu cgroup controller. Cpu // cpuacct cgroup controller. Cpuacct // cpuset cgroup controller. Cpuset // devices cgroup controller. Devices // freezer cgroup controller. Freezer // hugetlb cgroup controller. Hugetlb // memory cgroup controller. Memory // net_cls cgroup controller. NetCls // net_prio cgroup controller. NetPrio // per_event cgroup controller. PerfEvent // pids cgroup controller. Pids ) var ( // controllerNames maps controllers to names/relative paths. controllerNames = map[Controller]string{ Blkio: "blkio", Cpu: "cpu", Cpuacct: "cpuacct", Cpuset: "cpuset", Devices: "devices", Freezer: "freezer", Hugetlb: "hugetlb", Memory: "memory", NetCls: "net_cls", NetPrio: "net_prio", PerfEvent: "perf_event", Pids: "pids", } // controllerNames maps controllers to names/relative paths. controllerDirs = map[string]Controller{ "blkio": Blkio, "cpu": Cpu, "cpuacct": Cpuacct, "cpuset": Cpuset, "devices": Devices, "freezer": Freezer, "hugetlb": Hugetlb, "memory": Memory, "net_cls": NetCls, "net_prio": NetPrio, "perf_event": PerfEvent, "pids": Pids, } ) // String returns the name of the given controller. func (c Controller) String() string { if name, ok := controllerNames[c]; ok { return name } return "unknown" } // Path returns the absolute path of the given controller. func (c Controller) Path() string { return path.Join(mountDir, c.String()) } // RelPath returns the relative path of the given controller. func (c Controller) RelPath() string { return c.String() } // Group returns the given group for the controller. func (c Controller) Group(group string) Group { return Group(path.Join(mountDir, c.String(), group)) } // AsGroup returns the group for the given absolute directory path. func AsGroup(absDir string) Group { return Group(absDir) } // Controller returns the controller for the group. func (g Group) Controller() Controller { relPath := strings.TrimPrefix(string(g), mountDir+"/") split := strings.SplitN(relPath, "/", 2) if len(split) > 0 { return controllerDirs[split[0]] } return UnknownController } // GetTasks reads the pids of threads currently assigned to the group. func (g Group) GetTasks() ([]string, error) { return g.readPids(Tasks) } // GetProcesses reads the pids of processes currently assigned to the group. func (g Group) GetProcesses() ([]string, error) { return g.readPids(Procs) } // AddTasks writes the given thread pids to the group. func (g Group) AddTasks(pids ...string) error { return g.writePids(Tasks, pids...) } // AddProcesses writes the given process pids to the group. func (g Group) AddProcesses(pids ...string) error { return g.writePids(Procs, pids...) } // Write writes the formatted data to the groups entry. func (g Group) Write(entry, format string, args ...interface{}) error { entryPath := path.Join(string(g), entry) f, err := fsi.OpenFile(entryPath, os.O_WRONLY, 0644) if err != nil { return g.errorf("%q: failed to open for writing: %v", entry, err) } defer f.Close() data := fmt.Sprintf(format, args...) if _, err := f.Write([]byte(data)); err != nil { return g.errorf("%q: failed to write %q: %v", entry, data, err) } return nil } // Read the groups entry and return contents in a string func (g Group) Read(entry string) (string, error) { var buf bytes.Buffer entryPath := path.Join(string(g), entry) f, err := fsi.OpenFile(entryPath, os.O_RDONLY, 0644) if err != nil { return "", g.errorf("%q: failed to open for reading: %v", entry, err) } defer f.Close() if _, err := buf.ReadFrom(f); err != nil { return "", err } return buf.String(), nil } // readPids reads pids from a cgroup's tasks or procs entry. func (g Group) readPids(entry string) ([]string, error) { var pids []string pidFile := path.Join(string(g), entry) f, err := fsi.OpenFile(pidFile, os.O_RDONLY, 0644) if err != nil { return nil, g.errorf("failed to open %q: %v", entry, err) } defer f.Close() s := bufio.NewScanner(f) for s.Scan() { pids = append(pids, s.Text()) } if s.Err() != nil { return nil, g.errorf("failed to read %q: %v", entry, err) } return pids, nil } // writePids writes pids to a cgroup's tasks or procs entry. func (g Group) writePids(entry string, pids ...string) error { pidFile := path.Join(string(g), entry) f, err := fsi.OpenFile(pidFile, os.O_WRONLY, 0644) if err != nil { return g.errorf("failed to write pids to %q: %v", pidFile, err) } defer f.Close() for _, pid := range pids { if _, err := f.Write([]byte(pid)); err != nil { if !errors.Is(err, syscall.ESRCH) { return g.errorf("failed to write pid %s to %q: %v", pidFile, pid, err) } } } return nil } // error returns a formatted group-specific error. func (g Group) errorf(format string, args ...interface{}) error { name := strings.TrimPrefix(string(g), mountDir+"/") return fmt.Errorf("cgroup "+name+": "+format, args...) } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgroupcontrol_test.go000066400000000000000000000056231432612301500261100ustar00rootroot00000000000000package cgroups import ( "io" "syscall" "testing" "github.com/intel/goresctrl/pkg/testutils" ) var cpuacctMyGroupTasks string = "" var testfiles fsiIface = NewFsiMock(map[string]mockFile{ "/sys/fs/cgroup/blkio/kubepods/tasks": { data: []byte("1\n23\n4567890\n"), }, "/sys/fs/cgroup/cpu/open/permission/denied/cgroup.procs": { // simulate open permission denied open: func(string) (fileIface, error) { return nil, syscall.EACCES }, }, "/sys/fs/cgroup/cpuacct/store/all/writes/tasks": { // everything that is written can be read // (no overwrite / truncate) write: func(b []byte) (int, error) { cpuacctMyGroupTasks = cpuacctMyGroupTasks + string(b) + "\n" return len(b), nil }, read: func(b []byte) (int, error) { if len(cpuacctMyGroupTasks) == 0 { return 0, io.EOF } bytes := len(cpuacctMyGroupTasks) copy(b, []byte(cpuacctMyGroupTasks)) cpuacctMyGroupTasks = "" return bytes, nil }, }, "/sys/fs/cgroup/cpuset/read/io/error/tasks": { // every read causes I/O error read: func(b []byte) (int, error) { return 0, syscall.EIO }, }, "/sys/fs/cgroup/devices/write/io/error/cgroup.procs": { // every write causes I/O error write: func(b []byte) (int, error) { return 0, syscall.EIO }, }, }) func TestGetTasks(t *testing.T) { fsi = testfiles tasks, err := Blkio.Group("kubepods").GetTasks() testutils.VerifyNoError(t, err) testutils.VerifyStringSlices(t, []string{"1", "23", "4567890"}, tasks) } func TestGetProcesses(t *testing.T) { fsi = testfiles _, err := Cpu.Group("open/permission/denied").GetProcesses() testutils.VerifyError(t, err, 1, []string{"permission denied"}) } func TestAddTasks(t *testing.T) { fsi = testfiles if err := Cpuacct.Group("store/all/writes").AddTasks("0", "987654321"); !testutils.VerifyNoError(t, err) { return } if err := Cpuacct.Group("store/all/writes").AddTasks(); !testutils.VerifyNoError(t, err) { return } if err := Cpuacct.Group("store/all/writes").AddTasks("12"); !testutils.VerifyNoError(t, err) { return } tasks, err := Cpuacct.Group("store/all/writes").GetTasks() testutils.VerifyNoError(t, err) testutils.VerifyStringSlices(t, []string{"0", "987654321", "12"}, tasks) } func TestAddProcesses(t *testing.T) { fsi = testfiles err := Devices.Group("write/io/error").AddProcesses("1") testutils.VerifyError(t, err, 1, []string{"input/output error"}) err = Freezer.Group("file/not/found").AddProcesses("1") testutils.VerifyError(t, err, 1, []string{"file not found"}) } func TestAsGroup(t *testing.T) { memGroupIn := Memory.Group("my/memory") memGroupOut := AsGroup(string(memGroupIn)) testutils.VerifyStrings(t, string(memGroupIn), string(memGroupOut)) } func TestGroupToController(t *testing.T) { c := Hugetlb.Group("my/group").Controller() testutils.VerifyStrings(t, "hugetlb", c.String()) } func TestRelPath(t *testing.T) { relPath := NetCls.RelPath() testutils.VerifyStrings(t, "net_cls", relPath) } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgroupid.go000066400000000000000000000021411432612301500237550ustar00rootroot00000000000000package cgroups import ( "fmt" "os" "path/filepath" "sync" ) // CgroupID implements mapping kernel cgroup IDs to cgroupfs paths with transparent caching. type CgroupID struct { root string cache map[uint64]string sync.Mutex } // NewCgroupID creates a new CgroupID map/cache. func NewCgroupID(root string) *CgroupID { return &CgroupID{ root: root, cache: make(map[uint64]string), } } // Find finds the path for the given cgroup id. func (cgid *CgroupID) Find(id uint64) (string, error) { found := false var p string cgid.Lock() defer cgid.Unlock() if path, ok := cgid.cache[id]; ok { return path, nil } err := fsi.Walk(cgid.root, func(path string, info os.FileInfo, err error) error { if err != nil { if os.IsNotExist(err) { return nil } return err } if found { return filepath.SkipDir } if info.IsDir() && id == getID(path) { found = true p = path return filepath.SkipDir } return nil }) if err != nil { return "", err } else if !found { return "", fmt.Errorf("cgroupid %v not found", id) } else { cgid.cache[id] = p return p, nil } } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgroupid_linux.go000066400000000000000000000004261432612301500252000ustar00rootroot00000000000000//go:build linux // +build linux package cgroups import ( "encoding/binary" "golang.org/x/sys/unix" ) func getID(path string) uint64 { h, _, err := unix.NameToHandleAt(unix.AT_FDCWD, path, 0) if err != nil { return 0 } return binary.LittleEndian.Uint64(h.Bytes()) } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgroupid_other.go000066400000000000000000000001421432612301500251550ustar00rootroot00000000000000//go:build !linux // +build !linux package cgroups func getID(path string) uint64 { return 0 } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/cgrouppath.go000066400000000000000000000042121432612301500243160ustar00rootroot00000000000000// Copyright 2020 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package cgroups import ( "path" "path/filepath" ) //nolint const ( // Tasks is a cgroup's "tasks" entry. Tasks = "tasks" // Procs is cgroup's "cgroup.procs" entry. Procs = "cgroup.procs" // CpuShares is the cpu controller's "cpu.shares" entry. CpuShares = "cpu.shares" // CpuPeriod is the cpu controller's "cpu.cfs_period_us" entry. CpuPeriod = "cpu.cfs_period_us" // CpuQuota is the cpu controller's "cpu.cfs_quota_us" entry. CpuQuota = "cpu.cfs_quota_us" // CpusetCpus is the cpuset controller's cpuset.cpus entry. CpusetCpus = "cpuset.cpus" // CpusetMems is the cpuset controller's cpuset.mems entry. CpusetMems = "cpuset.mems" ) var ( // mount is the parent directory for per-controller cgroupfs mounts. mountDir = "/sys/fs/cgroup" // v2Dir is the parent directory for per-controller cgroupfs mounts. v2Dir = path.Join(mountDir, "unified") // KubeletRoot is the --cgroup-root option the kubelet is running with. KubeletRoot = "" ) // GetMountDir returns the common mount point for cgroup v1 controllers. func GetMountDir() string { return mountDir } // SetMountDir sets the common mount point for the cgroup v1 controllers. func SetMountDir(dir string) { v2, _ := filepath.Rel(mountDir, v2Dir) mountDir = dir if v2 != "" { v2Dir = path.Join(mountDir, v2) } } // GetV2Dir returns the cgroup v2 unified mount directory. func GetV2Dir() string { return v2Dir } // SetV2Dir sets the unified cgroup v2 mount directory. func SetV2Dir(dir string) { if dir[0] == '/' { v2Dir = dir } else { v2Dir = path.Join(mountDir, v2Dir) } } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/fsi.go000066400000000000000000000022241432612301500227240ustar00rootroot00000000000000// Copyright 2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // This module defines filesystem interface (fsi) through which // cgroups package accesses files. package cgroups import ( "os" "path/filepath" ) type fsiIface interface { Open(name string) (fileIface, error) OpenFile(name string, flag int, perm os.FileMode) (fileIface, error) Walk(string, filepath.WalkFunc) error Lstat(path string) (os.FileInfo, error) } type fileIface interface { Close() error Read(p []byte) (n int, err error) Write(b []byte) (n int, err error) } // Set the default filesystem interface var fsi fsiIface = newFsiOS() golang-github-intel-goresctrl-0.3.0/pkg/cgroups/fsimock.go000066400000000000000000000125441432612301500236040ustar00rootroot00000000000000// Copyright 2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // This module implements a mock filesystem that can be used as a // replacement for the native filesystem interface (fsi). package cgroups import ( "fmt" "io" "os" "path/filepath" "strings" "time" ) type fsMock struct { files map[string]*mockFile // filesystem contents } type mockFile struct { // User-defined file properties data []byte // contents of the file // User/fsimock-defined properties info *mockFileInfo // File-specific user-overrides for the default file behavior open func(string) (fileIface, error) read func([]byte) (int, error) write func([]byte) (int, error) // fsimock-defined properties fs *fsMock filename string handle *mockFileHandle writeHistory [][]byte } type mockFileHandle struct { pos int } type mockFileInfo struct { mode os.FileMode name string mf *mockFile } func NewFsiMock(files map[string]mockFile) fsiIface { mfs := fsMock{} mfs.files = map[string]*mockFile{} for filename, usermf := range files { mf := usermf if mf.info == nil { mf.info = &mockFileInfo{} } if mf.info.name == "" { mf.info.name = filepath.Base(filename) } mf.filename = filename mf.info.mf = &mf mf.fs = &mfs mfs.files[filename] = &mf } return &mfs } func (mfs fsMock) OpenFile(name string, flag int, perm os.FileMode) (fileIface, error) { fsmockLog("OpenFile(%q, %d, %d)", name, flag, perm) if mf, ok := mfs.files[name]; ok { mf.handle = &mockFileHandle{} if mf.open != nil { return mf.open(name) } return *mf, nil } return nil, fsmockErrorf("%q: file not found", name) } func (mfs fsMock) Open(name string) (fileIface, error) { return mfs.OpenFile(name, 0, 0) } func (mfs fsMock) Walk(path string, walkFn filepath.WalkFunc) error { dirPath := strings.TrimSuffix(path, "/") info, err := mfs.Lstat(dirPath) if err != nil { err = walkFn(path, nil, err) return err } if !info.IsDir() { return walkFn(path, info, nil) } err = walkFn(path, info, nil) if err != nil { return err } for _, name := range mfs.dirContents(dirPath) { if err = mfs.Walk(dirPath+"/"+name, walkFn); err != nil && err != filepath.SkipDir { return err } } return nil } func (mfs fsMock) dirContents(path string) []string { dirPathS := strings.TrimSuffix(path, "/") + "/" contentSet := map[string]struct{}{} for filename := range mfs.files { if !strings.HasPrefix(filename, dirPathS) { continue } relToDirPath := strings.TrimPrefix(filename, dirPathS) names := strings.SplitN(relToDirPath, "/", 2) contentSet[names[0]] = struct{}{} } contents := make([]string, 0, len(contentSet)) for name := range contentSet { contents = append(contents, name) } return contents } func (mfs fsMock) Lstat(path string) (os.FileInfo, error) { if mf, ok := mfs.files[path]; ok { return *mf.info, nil } if len(mfs.dirContents(path)) > 0 { return mockFileInfo{ name: filepath.Base(path), mode: os.ModeDir, }, nil } return mockFileInfo{}, fsmockErrorf("%q: file not found", path) } func (mfi mockFileInfo) Name() string { return mfi.name } func (mfi mockFileInfo) Size() int64 { if mfi.mf != nil { return int64(len(mfi.mf.data)) } return 0 } func (mfi mockFileInfo) Mode() os.FileMode { return mfi.mode } func (mfi mockFileInfo) ModTime() time.Time { return time.Time{} } func (mfi mockFileInfo) IsDir() bool { return mfi.mode&os.ModeDir != 0 } func (mfi mockFileInfo) Sys() interface{} { return nil } func (mf mockFile) Write(b []byte) (n int, err error) { pos := mf.handle.pos if mf.write != nil { n, err = mf.write(b) if err == nil { mf.fs.files[mf.filename].writeHistory = append(mf.fs.files[mf.filename].writeHistory, b) } } else { newpos := pos + len(b) if newpos > cap(mf.data) { newdata := make([]byte, newpos) copy(newdata, mf.data) mf.data = newdata } copy(mf.data[pos:newpos], b) mf.handle.pos = newpos if f, ok := mf.fs.files[mf.filename]; ok { f.data = mf.data } mf.fs.files[mf.filename].writeHistory = append(mf.fs.files[mf.filename].writeHistory, b) } fsmockLog("{%q, pos=%d}.Write([%d]byte(%q)) = (%d, %v) %q", mf.filename, pos, len(b), string(b), n, err, mf.fs.files[mf.filename].data) return n, err } func (mf mockFile) Read(b []byte) (n int, err error) { pos := mf.handle.pos if mf.read != nil { n, err = mf.read(b) } else { n = len(mf.data) - pos err = nil if n <= 0 { err = io.EOF } if n > cap(b) { n = cap(b) } copy(b, mf.data[pos:pos+n]) mf.handle.pos += n } fsmockLog("{%q, pos=%d}.Read([%d]byte) = (%d, %v)\n", mf.filename, pos, len(b), n, err) return } func (mf mockFile) Close() error { return nil } func fsmockLog(format string, args ...interface{}) { fmt.Printf("fsmock: "+format+"\n", args...) } func fsmockErrorf(format string, args ...interface{}) error { return fmt.Errorf("fsmock: "+format, args...) } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/fsimock_test.go000066400000000000000000000034021432612301500246340ustar00rootroot00000000000000package cgroups import ( "os" "path/filepath" "sort" "testing" "github.com/intel/goresctrl/pkg/testutils" ) var fsMockUtFiles map[string]mockFile = map[string]mockFile{ "/my/emptyfile": {}, "/my/emptydir": { info: &mockFileInfo{mode: os.ModeDir}, }, "/my/dir/data0": {data: []byte("abc")}, "/my/dir/data1": {data: []byte("xyz")}, } func TestWalk(t *testing.T) { fs := NewFsiMock(fsMockUtFiles) foundNotInMyDir := []string{} err := fs.Walk("/", func(path string, info os.FileInfo, err error) error { if filepath.Base(path) == "dir" { return filepath.SkipDir } foundNotInMyDir = append(foundNotInMyDir, path) return nil }) testutils.VerifyNoError(t, err) sort.Strings(foundNotInMyDir) testutils.VerifyStringSlices(t, []string{"/", "/my", "/my/emptydir", "/my/emptyfile"}, foundNotInMyDir) } func TestReadWrite(t *testing.T) { var info os.FileInfo fs := NewFsiMock(fsMockUtFiles) f, err := fs.OpenFile("/my/dir/data0", os.O_WRONLY, 0) testutils.VerifyNoError(t, err) _, err = f.Write([]byte{}) testutils.VerifyNoError(t, err) _, err = f.Write([]byte("01")) testutils.VerifyNoError(t, err) info, err = fs.Lstat("/my/dir/data0") testutils.VerifyNoError(t, err) if info.Size() != 3 { t.Errorf("expected file size %d, got %d", 3, info.Size()) } _, err = f.Write([]byte("23")) testutils.VerifyNoError(t, err) if info.Size() != 4 { t.Errorf("expected file size %d, got %d", 4, info.Size()) } f.Close() f, err = fs.OpenFile("/my/dir/data0", os.O_RDONLY, 0) testutils.VerifyNoError(t, err) buf := make([]byte, 10) bytes, err := f.Read(buf) testutils.VerifyNoError(t, err) if bytes != 4 { t.Errorf("expected to read %d bytes, Read returned %d", 4, bytes) } testutils.VerifyStringSlices(t, []string{"0123"}, []string{string(buf[:bytes])}) } golang-github-intel-goresctrl-0.3.0/pkg/cgroups/fsios.go000066400000000000000000000027641432612301500232770ustar00rootroot00000000000000// Copyright 2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // This module provides the native implementation of the filesystem // interface (fsi). package cgroups import ( "os" "path/filepath" ) type fsOs struct{} type osFile struct { file *os.File } func newFsiOS() fsiIface { return fsOs{} } func (fsOs) OpenFile(name string, flag int, perm os.FileMode) (fileIface, error) { f, err := os.OpenFile(name, flag, perm) return osFile{f}, err } func (fsOs) Open(name string) (fileIface, error) { f, err := os.Open(name) return osFile{f}, err } func (fsOs) Lstat(name string) (os.FileInfo, error) { return os.Lstat(name) } func (fsOs) Walk(root string, walkFn filepath.WalkFunc) error { return filepath.Walk(root, walkFn) } func (osf osFile) Write(b []byte) (n int, err error) { return osf.file.Write(b) } func (osf osFile) Read(b []byte) (n int, err error) { return osf.file.Read(b) } func (osf osFile) Close() error { return osf.file.Close() } golang-github-intel-goresctrl-0.3.0/pkg/kubernetes/000077500000000000000000000000001432612301500223015ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/kubernetes/annotations.go000066400000000000000000000036301432612301500251670ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package kubernetes // ClassOrigin type indicates the source of container's class // information: whether it is found from CRI level container // annotations, Kubernetes' pod annotations, or it has not been found // at all. type ClassOrigin int const ( ClassOriginNotFound ClassOrigin = iota ClassOriginContainerAnnotation ClassOriginPodAnnotation ) func (c ClassOrigin) String() string { switch c { case ClassOriginNotFound: return "" case ClassOriginContainerAnnotation: return "container annotations" case ClassOriginPodAnnotation: return "pod annotations" default: return "" } } // ContainerClassFromAnnotations determines the effective class of a // container from the Pod annotations and CRI level container // annotations of a container. func ContainerClassFromAnnotations(containerAnnotation, podAnnotation, podAnnotationContainerPrefix string, containerName string, containerAnnotations, podAnnotations map[string]string) (string, ClassOrigin) { if clsName, ok := containerAnnotations[containerAnnotation]; ok { return clsName, ClassOriginContainerAnnotation } if clsName, ok := podAnnotations[podAnnotationContainerPrefix+containerName]; ok { return clsName, ClassOriginPodAnnotation } if clsName, ok := podAnnotations[podAnnotation]; ok { return clsName, ClassOriginPodAnnotation } return "", ClassOriginNotFound } golang-github-intel-goresctrl-0.3.0/pkg/kubernetes/annotations_test.go000066400000000000000000000074221432612301500262310ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package kubernetes import ( "testing" ) // TestContainerClassFromAnnotations: unit test for ContainerClassFromAnnotations. func TestContainerClassFromAnnotations(t *testing.T) { aContainerAnnotation := "io.kubernetes.cri.a-class" aPodAnnotation := "a.resources.beta.kubernetes.io/pod" aPodAnnotationContainerPrefix := "a.resources.beta.kubernetes.io/container." bContainerAnnotation := "io.kubernetes.cri.b-class" bPodAnnotation := "b.resources.beta.kubernetes.io/pod" bPodAnnotationContainerPrefix := "b.resources.beta.kubernetes.io/container." allContainerAnnotations := map[string]string{ aContainerAnnotation: "a-container-class", bContainerAnnotation: "b-container-class", } allPodAnnotations := map[string]string{ aPodAnnotation: "a-pod-class", aPodAnnotationContainerPrefix + "special-container": "a-pod-container-class", bPodAnnotation: "b-pod-class", bPodAnnotationContainerPrefix + "special-container": "b-pod-container-class", } tcases := []struct { name string // the name of the test case // inputs lookForCA string // container annotation to look for lookForPA string // pod annotation to look for lookForPAC string // pod annotation container prefix to look for cName string // container name cAnns map[string]string // container annotations pAnns map[string]string // pod annotations // outputs expectedClass string expectedOrigin ClassOrigin }{ { name: "all empty", expectedOrigin: ClassOriginNotFound, }, { name: "container annotation overrides all pod annotations", lookForCA: aContainerAnnotation, lookForPA: aPodAnnotation, lookForPAC: aPodAnnotationContainerPrefix, cName: "special-container", cAnns: allContainerAnnotations, pAnns: allPodAnnotations, expectedClass: "a-container-class", expectedOrigin: ClassOriginContainerAnnotation, }, { name: "container prefix overrides default pod annotation", lookForCA: "not.existing.container.annotation", lookForPA: bPodAnnotation, lookForPAC: bPodAnnotationContainerPrefix, cName: "special-container", cAnns: allContainerAnnotations, pAnns: allPodAnnotations, expectedClass: "b-pod-container-class", expectedOrigin: ClassOriginPodAnnotation, }, { name: "default pod annotation", lookForCA: "not.existing.container.annotation", lookForPA: bPodAnnotation, lookForPAC: bPodAnnotationContainerPrefix, cName: "ordinary-container", cAnns: allContainerAnnotations, pAnns: allPodAnnotations, expectedClass: "b-pod-class", expectedOrigin: ClassOriginPodAnnotation, }, } for _, tc := range tcases { t.Run(tc.name, func(t *testing.T) { observedClass, observedOrigin := ContainerClassFromAnnotations( tc.lookForCA, tc.lookForPA, tc.lookForPAC, tc.cName, tc.cAnns, tc.pAnns) if observedClass != tc.expectedClass { t.Errorf("expected class %q, observed %q", tc.expectedClass, observedClass) } if observedOrigin != tc.expectedOrigin { t.Errorf("expected origin %q, observed %q", tc.expectedOrigin, observedOrigin) } }) } } golang-github-intel-goresctrl-0.3.0/pkg/log/000077500000000000000000000000001432612301500207135ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/log/log.go000066400000000000000000000043301432612301500220230ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package log import ( "fmt" stdlog "log" "strings" ) // Logger is the logging interface for goresctl type Logger interface { Debugf(format string, v ...interface{}) Infof(format string, v ...interface{}) Warnf(format string, v ...interface{}) Errorf(format string, v ...interface{}) Panicf(format string, v ...interface{}) Fatalf(format string, v ...interface{}) } type logger struct { *stdlog.Logger } // NewLoggerWrapper wraps an implementation of the golang standard intreface // into a goresctl specific compatible logger interface func NewLoggerWrapper(l *stdlog.Logger) Logger { return &logger{Logger: l} } func (l *logger) Debugf(format string, v ...interface{}) { l.Logger.Printf("DEBUG: "+format, v...) } func (l *logger) Infof(format string, v ...interface{}) { l.Logger.Printf("INFO: "+format, v...) } func (l *logger) Warnf(format string, v ...interface{}) { l.Logger.Printf("WARN: "+format, v...) } func (l *logger) Errorf(format string, v ...interface{}) { l.Logger.Printf("ERROR: "+format, v...) } func (l *logger) Panicf(format string, v ...interface{}) { l.Logger.Panicf(format, v...) } func (l *logger) Fatalf(format string, v ...interface{}) { l.Logger.Fatalf(format, v...) } func InfoBlock(l Logger, heading, linePrefix, format string, v ...interface{}) { l.Infof("%s", heading) lines := strings.Split(fmt.Sprintf(format, v...), "\n") for _, line := range lines { l.Infof("%s%s", linePrefix, line) } } func DebugBlock(l Logger, heading, linePrefix, format string, v ...interface{}) { l.Debugf("%s", heading) lines := strings.Split(fmt.Sprintf(format, v...), "\n") for _, line := range lines { l.Debugf("%s%s", linePrefix, line) } } golang-github-intel-goresctrl-0.3.0/pkg/rdt/000077500000000000000000000000001432612301500207235ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/rdt/bitmask.go000066400000000000000000000052351432612301500227110ustar00rootroot00000000000000/* Copyright 2019 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdt import ( "fmt" "math/bits" "strconv" "strings" ) // bitmask represents a generic 64 bit wide bitmask type bitmask uint64 // MarshalJSON implements the Marshaler interface of "encoding/json" func (b bitmask) MarshalJSON() ([]byte, error) { return []byte(fmt.Sprintf("\"%#x\"", b)), nil } // listStr prints the bitmask in human-readable format, similar to e.g. the // cpuset format of the Linux kernel func (b bitmask) listStr() string { str := "" sep := "" shift := int(0) lsbOne := b.lsbOne() // Process "ranges of ones" for lsbOne != -1 { b >>= uint(lsbOne) // Get range lenght from the position of the first zero numOnes := b.lsbZero() if numOnes == 1 { str += sep + strconv.Itoa(lsbOne+shift) } else { str += sep + strconv.Itoa(lsbOne+shift) + "-" + strconv.Itoa(lsbOne+numOnes-1+shift) } // Shift away the bits that have been processed b >>= uint(numOnes) shift += lsbOne + numOnes // Get next bit that is set (if any) lsbOne = b.lsbOne() sep = "," } return str } // listStrToBitmask parses a string containing a human-readable list of bit // numbers into a bitmask func listStrToBitmask(str string) (bitmask, error) { b := bitmask(0) // Empty bitmask if len(str) == 0 { return b, nil } ranges := strings.Split(str, ",") for _, ran := range ranges { split := strings.SplitN(ran, "-", 2) bitNum, err := strconv.ParseUint(split[0], 10, 6) if err != nil { return b, fmt.Errorf("invalid bitmask %q: %v", str, err) } if len(split) == 1 { b |= 1 << bitNum } else { endNum, err := strconv.ParseUint(split[1], 10, 6) if err != nil { return b, fmt.Errorf("invalid bitmask %q: %v", str, err) } if endNum <= bitNum { return b, fmt.Errorf("invalid range %q in bitmask %q", ran, str) } b |= (1<<(endNum-bitNum+1) - 1) << bitNum } } return b, nil } func (b bitmask) lsbOne() int { if b == 0 { return -1 } return bits.TrailingZeros64(uint64(b)) } func (b bitmask) msbOne() int { // Returns -1 for b == 0 return 63 - bits.LeadingZeros64(uint64(b)) } func (b bitmask) lsbZero() int { return bits.TrailingZeros64(^uint64(b)) } golang-github-intel-goresctrl-0.3.0/pkg/rdt/config.go000066400000000000000000001024441432612301500225240ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdt import ( "encoding/json" "fmt" "math" "math/bits" "sort" "strconv" "strings" grclog "github.com/intel/goresctrl/pkg/log" "github.com/intel/goresctrl/pkg/utils" ) // Config is the user-specified RDT configuration. type Config struct { Options Options `json:"options"` Partitions map[string]struct { L2Allocation CatConfig `json:"l2Allocation"` L3Allocation CatConfig `json:"l3Allocation"` MBAllocation MbaConfig `json:"mbAllocation"` Classes map[string]struct { L2Allocation CatConfig `json:"l2Allocation"` L3Allocation CatConfig `json:"l3Allocation"` MBAllocation MbaConfig `json:"mbAllocation"` Kubernetes KubernetesOptions `json:"kubernetes"` } `json:"classes"` } `json:"partitions"` } // CatConfig contains the L2 or L3 cache allocation configuration for one partition or class. type CatConfig map[string]CacheIdCatConfig // MbaConfig contains the memory bandwidth configuration for one partition or class. type MbaConfig map[string]CacheIdMbaConfig // CacheIdCatConfig is the cache allocation configuration for one cache id. // Code and Data represent an optional configuration for separate code and data // paths and only have effect when RDT CDP (Code and Data Prioritization) is // enabled in the system. Code and Data go in tandem so that both or neither // must be specified - only specifying the other is considered a configuration // error. type CacheIdCatConfig struct { Unified CacheProportion Code CacheProportion Data CacheProportion } // CacheIdMbaConfig is the memory bandwidth configuration for one cache id. // It's an array of at most two values, specifying separate values to be used // for percentage based and MBps based memory bandwidth allocation. For // example, `{"80%", "1000MBps"}` would allocate 80% if percentage based // allocation is used by the Linux kernel, or 1000 MBps in case MBps based // allocation is in use. type CacheIdMbaConfig []MbProportion // MbProportion specifies a share of available memory bandwidth. It's an // integer value followed by a unit. Two units are supported: // // - percentage, e.g. `80%` // - MBps, e.g. `1000MBps` type MbProportion string // CacheProportion specifies a share of the available cache lines. // Supported formats: // // - percentage, e.g. `50%` // - percentage range, e.g. `50-60%` // - bit numbers, e.g. `0-5`, `2,3`, must contain one contiguous block of bits set // - hex bitmask, e.g. `0xff0`, must contain one contiguous block of bits set type CacheProportion string // CacheIdAll is a special cache id used to denote a default, used as a // fallback for all cache ids that are not explicitly specified. const CacheIdAll = "all" // config represents the final (parsed and resolved) runtime configuration of // RDT Control type config struct { Options Options Partitions partitionSet Classes classSet } // partitionSet represents the pool of rdt partitions type partitionSet map[string]*partitionConfig // classSet represents the pool of rdt classes type classSet map[string]*classConfig // partitionConfig is the final configuration of one partition type partitionConfig struct { CAT map[cacheLevel]catSchema MB mbSchema } // classConfig represents configuration of one class, i.e. one CTRL group in // the Linux resctrl interface type classConfig struct { Partition string CATSchema map[cacheLevel]catSchema MBSchema mbSchema Kubernetes KubernetesOptions } // Options contains common settings. type Options struct { L2 CatOptions `json:"l2"` L3 CatOptions `json:"l3"` MB MbOptions `json:"mb"` } // CatOptions contains the common settings for cache allocation. type CatOptions struct { Optional bool } // MbOptions contains the common settings for memory bandwidth allocation. type MbOptions struct { Optional bool } // KubernetesOptions contains per-class settings for the Kubernetes-related functionality. type KubernetesOptions struct { DenyPodAnnotation bool `json:"denyPodAnnotation"` DenyContainerAnnotation bool `json:"denyContainerAnnotation"` } // catSchema represents a cache part of the schemata of a class (i.e. resctrl group) type catSchema struct { Lvl cacheLevel Alloc catSchemaRaw } // catSchemaRaw is the cache schemata without the information about cache level type catSchemaRaw map[uint64]catAllocation // mbSchema represents the MB part of the schemata of a class (i.e. resctrl group) type mbSchema map[uint64]uint64 // catAllocation describes the allocation configuration for one cache id type catAllocation struct { Unified cacheAllocation Code cacheAllocation `json:",omitempty"` Data cacheAllocation `json:",omitempty"` } // cacheAllocation is the basic interface for handling cache allocations of one // type (unified, code, data) type cacheAllocation interface { Overlay(bitmask, uint64) (bitmask, error) } // catAbsoluteAllocation represents an explicitly specified cache allocation // bitmask type catAbsoluteAllocation bitmask // catPctAllocation represents a relative (percentage) share of the available // bitmask type catPctAllocation uint64 // catPctRangeAllocation represents a percentage range of the available bitmask type catPctRangeAllocation struct { lowPct uint64 highPct uint64 } // catSchemaType represents different L3 cache allocation schemes type catSchemaType string const ( // catSchemaTypeUnified is the schema type when CDP is not enabled catSchemaTypeUnified catSchemaType = "unified" // catSchemaTypeCode is the 'code' part of CDP schema catSchemaTypeCode catSchemaType = "code" // catSchemaTypeData is the 'data' part of CDP schema catSchemaTypeData catSchemaType = "data" ) // cat returns CAT options for the specified cache level. func (o Options) cat(lvl cacheLevel) CatOptions { switch lvl { case L2: return o.L2 case L3: return o.L3 } return CatOptions{} } func (t catSchemaType) toResctrlStr() string { if t == catSchemaTypeUnified { return "" } return strings.ToUpper(string(t)) } const ( mbSuffixPct = "%" mbSuffixMbps = "MBps" ) func newCatSchema(typ cacheLevel) catSchema { return catSchema{ Lvl: typ, Alloc: make(map[uint64]catAllocation), } } // toStr returns the CAT schema in a format accepted by the Linux kernel // resctrl (schemata) interface func (s catSchema) toStr(typ catSchemaType, baseSchema catSchema) (string, error) { schema := string(s.Lvl) + typ.toResctrlStr() + ":" sep := "" // Get a sorted slice of cache ids for deterministic output ids := append([]uint64{}, info.cat[s.Lvl].cacheIds...) utils.SortUint64s(ids) minBits := info.cat[s.Lvl].minCbmBits() for _, id := range ids { // Default to 100% bmask := info.cat[s.Lvl].cbmMask() if base, ok := baseSchema.Alloc[id]; ok { baseMask, ok := base.getEffective(typ).(catAbsoluteAllocation) if !ok { return "", fmt.Errorf("BUG: basemask not of type catAbsoluteAllocation") } bmask = bitmask(baseMask) } if s.Alloc != nil { var err error masks := s.Alloc[id] overlayMask := masks.getEffective(typ) bmask, err = overlayMask.Overlay(bmask, minBits) if err != nil { return "", err } } schema += fmt.Sprintf("%s%d=%x", sep, id, bmask) sep = ";" } return schema + "\n", nil } func (a catAllocation) get(typ catSchemaType) cacheAllocation { switch typ { case catSchemaTypeCode: return a.Code case catSchemaTypeData: return a.Data } return a.Unified } func (a catAllocation) set(typ catSchemaType, v cacheAllocation) catAllocation { switch typ { case catSchemaTypeCode: a.Code = v case catSchemaTypeData: a.Data = v default: a.Unified = v } return a } func (a catAllocation) getEffective(typ catSchemaType) cacheAllocation { switch typ { case catSchemaTypeCode: if a.Code != nil { return a.Code } case catSchemaTypeData: if a.Data != nil { return a.Data } } // Use Unified as the default/fallback for Code and Data return a.Unified } // Overlay function of the cacheAllocation interface func (a catAbsoluteAllocation) Overlay(baseMask bitmask, minBits uint64) (bitmask, error) { if err := verifyCatBaseMask(baseMask, minBits); err != nil { return 0, err } shiftWidth := baseMask.lsbOne() // Treat our bitmask relative to the basemask bmask := bitmask(a) << shiftWidth // Do bounds checking that we're "inside" the base mask if bmask|baseMask != baseMask { return 0, fmt.Errorf("bitmask %#x (%#x << %d) does not fit basemask %#x", bmask, a, shiftWidth, baseMask) } return bmask, nil } // MarshalJSON implements the Marshaler interface of "encoding/json" func (a catAbsoluteAllocation) MarshalJSON() ([]byte, error) { return []byte(fmt.Sprintf("\"%#x\"", a)), nil } // Overlay function of the cacheAllocation interface func (a catPctAllocation) Overlay(baseMask bitmask, minBits uint64) (bitmask, error) { return catPctRangeAllocation{highPct: uint64(a)}.Overlay(baseMask, minBits) } // Overlay function of the cacheAllocation interface func (a catPctRangeAllocation) Overlay(baseMask bitmask, minBits uint64) (bitmask, error) { if err := verifyCatBaseMask(baseMask, minBits); err != nil { return 0, err } baseMaskMsb := uint64(baseMask.msbOne()) baseMaskLsb := uint64(baseMask.lsbOne()) baseMaskNumBits := baseMaskMsb - baseMaskLsb + 1 low, high := a.lowPct, a.highPct if low == 0 { low = 1 } if low > high || low > 100 || high > 100 { return 0, fmt.Errorf("invalid percentage range in %v", a) } // Convert percentage limits to bit numbers // Our effective range is 1%-100%, use substraction (-1) because of // arithmetics, so that we don't overflow on 100% lsb := (low - 1) * baseMaskNumBits / 100 msb := (high - 1) * baseMaskNumBits / 100 // Make sure the number of bits set satisfies the minimum requirement numBits := msb - lsb + 1 if numBits < minBits { gap := minBits - numBits // First, widen the mask from the "lsb end" if gap <= lsb { lsb -= gap gap = 0 } else { gap -= lsb lsb = 0 } // If needed, widen the mask from the "msb end" msbAvailable := baseMaskNumBits - msb - 1 if gap <= msbAvailable { msb += gap } else { return 0, fmt.Errorf("BUG: not enough bits available for cache bitmask (%v applied on basemask %#x)", a, baseMask) } } value := ((1 << (msb - lsb + 1)) - 1) << (lsb + baseMaskLsb) return bitmask(value), nil } func verifyCatBaseMask(baseMask bitmask, minBits uint64) error { if baseMask == 0 { return fmt.Errorf("empty basemask not allowed") } // Check that the basemask contains one (and only one) contiguous block of // (enough) bits set baseMaskWidth := baseMask.msbOne() - baseMask.lsbOne() + 1 if bits.OnesCount64(uint64(baseMask)) != baseMaskWidth { return fmt.Errorf("invalid basemask %#x: more than one block of bits set", baseMask) } if uint64(bits.OnesCount64(uint64(baseMask))) < minBits { return fmt.Errorf("invalid basemask %#x: fewer than %d bits set", baseMask, minBits) } return nil } // MarshalJSON implements the Marshaler interface of "encoding/json" func (a catPctAllocation) MarshalJSON() ([]byte, error) { return []byte(fmt.Sprintf("\"%d%%\"", a)), nil } // MarshalJSON implements the Marshaler interface of "encoding/json" func (a catPctRangeAllocation) MarshalJSON() ([]byte, error) { return []byte(fmt.Sprintf("\"%d-%d%%\"", a.lowPct, a.highPct)), nil } // toStr returns the MB schema in a format accepted by the Linux kernel // resctrl (schemata) interface func (s mbSchema) toStr(base map[uint64]uint64) string { schema := "MB:" sep := "" // Get a sorted slice of cache ids for deterministic output ids := append([]uint64{}, info.mb.cacheIds...) utils.SortUint64s(ids) for _, id := range ids { baseAllocation, ok := base[id] if !ok { if info.mb.mbpsEnabled { baseAllocation = math.MaxUint32 } else { baseAllocation = 100 } } value := uint64(0) if info.mb.mbpsEnabled { value = math.MaxUint32 if s != nil { value = s[id] } // Limit to given base value if value > baseAllocation { value = baseAllocation } } else { allocation := uint64(100) if s != nil { allocation = s[id] } value = allocation * baseAllocation / 100 // Guarantee minimum bw so that writing out the schemata does not fail if value < info.mb.minBandwidth { value = info.mb.minBandwidth } } schema += fmt.Sprintf("%s%d=%d", sep, id, value) sep = ";" } return schema + "\n" } // listStrToArray parses a string containing a human-readable list of numbers // into an integer array func listStrToArray(str string) ([]int, error) { a := []int{} // Empty list if len(str) == 0 { return a, nil } ranges := strings.Split(str, ",") for _, ran := range ranges { split := strings.SplitN(ran, "-", 2) // We limit to 8 bits in order to avoid accidental super long slices num, err := strconv.ParseInt(split[0], 10, 8) if err != nil { return a, fmt.Errorf("invalid integer %q: %v", str, err) } if len(split) == 1 { a = append(a, int(num)) } else { endNum, err := strconv.ParseInt(split[1], 10, 8) if err != nil { return a, fmt.Errorf("invalid integer in range %q: %v", str, err) } if endNum <= num { return a, fmt.Errorf("invalid integer range %q in %q", ran, str) } for i := num; i <= endNum; i++ { a = append(a, int(i)) } } } sort.Ints(a) return a, nil } // resolve tries to resolve the requested configuration into a working // configuration func (c *Config) resolve() (config, error) { var err error conf := config{Options: c.Options} grclog.DebugBlock(log, "resolving configuration:", " ", "%s", utils.DumpJSON(c)) conf.Partitions, err = c.resolvePartitions() if err != nil { return conf, err } conf.Classes, err = c.resolveClasses() return conf, err } // resolvePartitions tries to resolve the requested resource allocations of // partitions func (c *Config) resolvePartitions() (partitionSet, error) { // Initialize empty partition configuration conf := make(partitionSet, len(c.Partitions)) for name := range c.Partitions { conf[name] = &partitionConfig{ CAT: map[cacheLevel]catSchema{ L2: newCatSchema(L2), L3: newCatSchema(L3), }, MB: make(mbSchema, len(info.mb.cacheIds))} } // Resolve L2 partition allocations err := c.resolveCatPartitions(L2, conf) if err != nil { return nil, err } // Try to resolve L3 partition allocations err = c.resolveCatPartitions(L3, conf) if err != nil { return nil, err } // Try to resolve MB partition allocations err = c.resolveMBPartitions(conf) if err != nil { return nil, err } return conf, nil } // resolveCatPartitions tries to resolve requested cache allocations between partitions func (c *Config) resolveCatPartitions(lvl cacheLevel, conf partitionSet) error { if len(c.Partitions) == 0 { return nil } // Resolve partitions in sorted order for reproducibility names := make([]string, 0, len(c.Partitions)) for name := range c.Partitions { names = append(names, name) } sort.Strings(names) resolver := newCacheResolver(lvl, names) // Parse requested allocations from user config and load the resolver for _, name := range names { var allocations catSchema var err error switch lvl { case L2: allocations, err = c.Partitions[name].L2Allocation.toSchema(L2) case L3: allocations, err = c.Partitions[name].L3Allocation.toSchema(L3) } if err != nil { return fmt.Errorf("failed to parse %s allocation request for partition %q: %v", lvl, name, err) } resolver.requests[name] = allocations.Alloc } // Run resolver fo partition allocations grants, err := resolver.resolve() if err != nil { return err } if grants == nil { log.Debugf("%s allocation disabled for all partitions", lvl) return nil } for name, grant := range grants { conf[name].CAT[lvl] = grant } heading := fmt.Sprintf("actual (and requested) %s allocations per partition and cache id:", lvl) infoStr := "" for name, partition := range resolver.requests { infoStr += name + "\n" for _, id := range resolver.ids { infoStr += fmt.Sprintf(" %2d: ", id) allocationReq := partition[id] for _, typ := range []catSchemaType{catSchemaTypeUnified, catSchemaTypeCode, catSchemaTypeData} { infoStr += string(typ) + " " requested := allocationReq.get(typ) switch v := requested.(type) { case catAbsoluteAllocation: infoStr += fmt.Sprintf(" ", v) case catPctAllocation: granted := grants[name].Alloc[id].get(typ).(catAbsoluteAllocation) requestedPct := fmt.Sprintf("(%d%%)", v) truePct := float64(bits.OnesCount64(uint64(granted))) * 100 / float64(resolver.bitsTotal) infoStr += fmt.Sprintf("%5.1f%% %-6s ", truePct, requestedPct) case nil: infoStr += " " } } infoStr += "\n" } } grclog.DebugBlock(log, heading, " ", "%s", infoStr) return nil } // cacheResolver is a helper for resolving exclusive (partition) cache // allocation requests type cacheResolver struct { lvl cacheLevel ids []uint64 minBits uint64 bitsTotal uint64 partitions []string requests map[string]catSchemaRaw grants map[string]catSchema } func newCacheResolver(lvl cacheLevel, partitions []string) *cacheResolver { r := &cacheResolver{ lvl: lvl, ids: info.cat[lvl].cacheIds, minBits: info.cat[lvl].minCbmBits(), bitsTotal: uint64(info.cat[lvl].cbmMask().lsbZero()), partitions: partitions, requests: make(map[string]catSchemaRaw, len(partitions)), grants: make(map[string]catSchema, len(partitions))} for _, p := range partitions { r.grants[p] = catSchema{Lvl: lvl, Alloc: make(catSchemaRaw, len(r.ids))} } return r } func (r *cacheResolver) resolve() (map[string]catSchema, error) { for _, id := range r.ids { err := r.resolveID(id) if err != nil { return nil, err } } return r.grants, nil } // resolveCacheID resolves the partition allocations for one cache id func (r *cacheResolver) resolveID(id uint64) error { for _, typ := range []catSchemaType{catSchemaTypeUnified, catSchemaTypeCode, catSchemaTypeData} { log.Debugf("resolving partitions for %q schema for cache id %d", typ, id) err := r.resolveType(id, typ) if err != nil { return err } } return nil } // resolveType resolve one schema type for one cache id func (r *cacheResolver) resolveType(id uint64, typ catSchemaType) error { // Sanity check: if any partition has l3 allocation of this schema type // configured check that all other partitions have it, too nils := []string{} for _, partition := range r.partitions { if r.requests[partition][id].get(typ) == nil { nils = append(nils, partition) } } if len(nils) > 0 && len(nils) != len(r.partitions) { return fmt.Errorf("some partitions (%s) missing %s %q allocation request for cache id %d", strings.Join(nils, ", "), r.lvl, typ, id) } // Act depending on the type of the first request in the list a := r.requests[r.partitions[0]][id].get(typ) switch a.(type) { case catAbsoluteAllocation: return r.resolveAbsolute(id, typ) case nil: default: return r.resolveRelative(id, typ) } return nil } func (r *cacheResolver) resolveRelative(id uint64, typ catSchemaType) error { type reqHelper struct { name string req uint64 } // Sanity check: // 1. allocation requests are of the same type (relative) // 2. total allocation requested for this cache id does not exceed 100 percent // Additionally fill a helper structure for sorting partitions percentageTotal := uint64(0) reqs := make([]reqHelper, 0, len(r.partitions)) for _, partition := range r.partitions { switch a := r.requests[partition][id].get(typ).(type) { case catPctAllocation: percentageTotal += uint64(a) reqs = append(reqs, reqHelper{name: partition, req: uint64(a)}) case catAbsoluteAllocation: return fmt.Errorf("error resolving %s allocation for cache id %d: mixing "+ "relative and absolute allocations between partitions not supported", r.lvl, id) case catPctRangeAllocation: return fmt.Errorf("percentage ranges in partition allocation not supported") default: return fmt.Errorf("BUG: unknown cacheAllocation type %T", a) } } if percentageTotal < 100 { log.Infof("requested total %s %q partition allocation for cache id %d <100%% (%d%%)", r.lvl, typ, id, percentageTotal) } else if percentageTotal > 100 { return fmt.Errorf("accumulated %s %q partition allocation requests for cache id %d exceeds 100%% (%d%%)", r.lvl, typ, id, percentageTotal) } // Sort partition allocations. We want to resolve smallest allocations // first in order to try to ensure that all allocations can be satisfied // because small percentages might need to be rounded up sort.Slice(reqs, func(i, j int) bool { return reqs[i].req < reqs[j].req }) // Calculate number of bits granted to each partition. grants := make(map[string]uint64, len(r.partitions)) bitsTotal := percentageTotal * uint64(r.bitsTotal) / 100 bitsAvailable := bitsTotal for i, req := range reqs { percentageAvailable := bitsAvailable * percentageTotal / bitsTotal // This might happen e.g. if number of partitions would be greater // than the total number of bits if bitsAvailable < r.minBits { return fmt.Errorf("unable to resolve %s allocation for cache id %d, not enough exlusive bits available", r.lvl, id) } // Use integer arithmetics, effectively always rounding down // fractional allocations i.e. trying to avoid over-allocation numBits := req.req * bitsAvailable / percentageAvailable // Guarantee a non-zero allocation if numBits < r.minBits { numBits = r.minBits } // Don't overflow, allocate all remaining bits to the last partition if numBits > bitsAvailable || i == len(reqs)-1 { numBits = bitsAvailable } grants[req.name] = numBits bitsAvailable -= numBits } // Construct the actual bitmasks for each partition lsbID := uint64(0) for _, partition := range r.partitions { // Compose the actual bitmask v := r.grants[partition].Alloc[id].set(typ, catAbsoluteAllocation(bitmask(((1< 0 { return fmt.Errorf("overlapping %s partition allocation requests for cache id %d", r.lvl, id) } mask |= bitmask(a) r.grants[partition].Alloc[id] = r.grants[partition].Alloc[id].set(typ, a) } return nil } // resolveMBPartitions tries to resolve requested MB allocations between partitions func (c *Config) resolveMBPartitions(conf partitionSet) error { // We use percentage values directly from the user conf for name, partition := range c.Partitions { allocations, err := partition.MBAllocation.toSchema() if err != nil { return fmt.Errorf("failed to resolve MB allocation for partition %q: %v", name, err) } for id, allocation := range allocations { conf[name].MB[id] = allocation // Check that we don't go under the minimum allowed bandwidth setting if !info.mb.mbpsEnabled && allocation < info.mb.minBandwidth { conf[name].MB[id] = info.mb.minBandwidth } } } return nil } // resolveClasses tries to resolve class allocations of all partitions func (c *Config) resolveClasses() (classSet, error) { classes := make(classSet) for bname, partition := range c.Partitions { for gname, class := range partition.Classes { gname = unaliasClassName(gname) if !IsQualifiedClassName(gname) { return classes, fmt.Errorf("unqualified class name %q (must not be '.' or '..' and must not contain '/' or newline)", gname) } if _, ok := classes[gname]; ok { return classes, fmt.Errorf("class names must be unique, %q defined multiple times", gname) } var err error gc := &classConfig{Partition: bname, CATSchema: make(map[cacheLevel]catSchema), Kubernetes: class.Kubernetes} gc.CATSchema[L2], err = class.L2Allocation.toSchema(L2) if err != nil { return classes, fmt.Errorf("failed to resolve L2 allocation for class %q: %v", gname, err) } if gc.CATSchema[L2].Alloc != nil && partition.L2Allocation == nil { return classes, fmt.Errorf("L2 allocation missing from partition %q but class %q specifies L2 schema", bname, gname) } gc.CATSchema[L3], err = class.L3Allocation.toSchema(L3) if err != nil { return classes, fmt.Errorf("failed to resolve L3 allocation for class %q: %v", gname, err) } if gc.CATSchema[L3].Alloc != nil && partition.L3Allocation == nil { return classes, fmt.Errorf("L3 allocation missing from partition %q but class %q specifies L3 schema", bname, gname) } gc.MBSchema, err = class.MBAllocation.toSchema() if err != nil { return classes, fmt.Errorf("failed to resolve MB allocation for class %q: %v", gname, err) } if gc.MBSchema != nil && partition.MBAllocation == nil { return classes, fmt.Errorf("MB allocation missing from partition %q but class %q specifies MB schema", bname, gname) } classes[gname] = gc } } return classes, nil } // toSchema converts a cache allocation config to effective allocation schema covering all cache IDs func (c CatConfig) toSchema(lvl cacheLevel) (catSchema, error) { if c == nil { return catSchema{Lvl: lvl}, nil } allocations := newCatSchema(lvl) minBits := info.cat[lvl].minCbmBits() d, ok := c[CacheIdAll] if !ok { d = CacheIdCatConfig{Unified: "100%"} } defaultVal, err := d.parse(minBits) if err != nil { return allocations, err } // Pre-fill with defaults for _, i := range info.cat[lvl].cacheIds { allocations.Alloc[i] = defaultVal } for key, val := range c { if key == CacheIdAll { continue } ids, err := listStrToArray(key) if err != nil { return allocations, err } schemaVal, err := val.parse(minBits) if err != nil { return allocations, err } for _, id := range ids { if _, ok := allocations.Alloc[uint64(id)]; ok { allocations.Alloc[uint64(id)] = schemaVal } } } return allocations, nil } // catConfig is a helper for unmarshalling CatConfig type catConfig CatConfig // UnmarshalJSON implements the Unmarshaler interface of "encoding/json" func (c *CatConfig) UnmarshalJSON(data []byte) error { raw := new(interface{}) err := json.Unmarshal(data, raw) if err != nil { return err } conf := CatConfig{} switch v := (*raw).(type) { case string: conf[CacheIdAll] = CacheIdCatConfig{Unified: CacheProportion(v)} default: // Use the helper type to avoid infinite recursion helper := catConfig{} if err := json.Unmarshal(data, &helper); err != nil { return err } for k, v := range helper { conf[k] = v } } *c = conf return nil } // toSchema converts an MB allocation config to effective allocation schema covering all cache IDs func (c MbaConfig) toSchema() (mbSchema, error) { if c == nil { return nil, nil } d, ok := c[CacheIdAll] if !ok { d = CacheIdMbaConfig{"100" + mbSuffixPct, "4294967295" + mbSuffixMbps} } defaultVal, err := d.parse() if err != nil { return nil, err } allocations := make(mbSchema, len(info.mb.cacheIds)) // Pre-fill with defaults for _, i := range info.mb.cacheIds { allocations[i] = defaultVal } for key, val := range c { if key == CacheIdAll { continue } ids, err := listStrToArray(key) if err != nil { return nil, err } schemaVal, err := val.parse() if err != nil { return nil, err } for _, id := range ids { if _, ok := allocations[uint64(id)]; ok { allocations[uint64(id)] = schemaVal } } } return allocations, nil } // mbaConfig is a helper for unmarshalling MbaConfig type mbaConfig MbaConfig // UnmarshalJSON implements the Unmarshaler interface of "encoding/json" func (c *MbaConfig) UnmarshalJSON(data []byte) error { raw := new(interface{}) err := json.Unmarshal(data, raw) if err != nil { return err } conf := MbaConfig{} switch (*raw).(type) { case []interface{}: helper := CacheIdMbaConfig{} if err := json.Unmarshal(data, &helper); err != nil { return err } conf[CacheIdAll] = helper default: // Use the helper type to avoid infinite recursion helper := mbaConfig{} if err := json.Unmarshal(data, &helper); err != nil { return err } for k, v := range helper { conf[k] = v } } *c = conf return nil } // parse per cache-id CAT configuration into an effective allocation to be used // in the CAT schema func (c *CacheIdCatConfig) parse(minBits uint64) (catAllocation, error) { var err error allocation := catAllocation{} allocation.Unified, err = c.Unified.parse(minBits) if err != nil { return allocation, err } allocation.Code, err = c.Code.parse(minBits) if err != nil { return allocation, err } allocation.Data, err = c.Data.parse(minBits) if err != nil { return allocation, err } // Sanity check for the configuration if allocation.Unified == nil { return allocation, fmt.Errorf("'unified' not specified in cache schema %s", *c) } if allocation.Code != nil && allocation.Data == nil { return allocation, fmt.Errorf("'code' specified but missing 'data' from cache schema %s", *c) } if allocation.Code == nil && allocation.Data != nil { return allocation, fmt.Errorf("'data' specified but missing 'code' from cache schema %s", *c) } return allocation, nil } // cacheIdCatConfig is a helper for unmarshalling CacheIdCatConfig type cacheIdCatConfig CacheIdCatConfig // UnmarshalJSON implements the Unmarshaler interface of "encoding/json" func (c *CacheIdCatConfig) UnmarshalJSON(data []byte) error { raw := new(interface{}) err := json.Unmarshal(data, raw) if err != nil { return err } conf := CacheIdCatConfig{} switch v := (*raw).(type) { case string: conf.Unified = CacheProportion(v) default: // Use the helper type to avoid infinite recursion helper := cacheIdCatConfig{} if err := json.Unmarshal(data, &helper); err != nil { return err } conf.Unified = helper.Unified conf.Code = helper.Code conf.Data = helper.Data } *c = conf return nil } // parse converts a per cache-id MBA configuration into effective value // to be used in the MBA schema func (c *CacheIdMbaConfig) parse() (uint64, error) { for _, v := range *c { str := string(v) if strings.HasSuffix(str, mbSuffixPct) { if !info.mb.mbpsEnabled { value, err := strconv.ParseUint(strings.TrimSuffix(str, mbSuffixPct), 10, 7) if err != nil { return 0, err } return value, nil } } else if strings.HasSuffix(str, mbSuffixMbps) { if info.mb.mbpsEnabled { value, err := strconv.ParseUint(strings.TrimSuffix(str, mbSuffixMbps), 10, 32) if err != nil { return 0, err } return value, nil } } else { log.Warnf("unrecognized MBA allocation unit in %q", str) } } // No value for the active mode was specified if info.mb.mbpsEnabled { return 0, fmt.Errorf("missing 'MBps' value from mbSchema; required because 'mba_MBps' is enabled in the system") } return 0, fmt.Errorf("missing '%%' value from mbSchema; required because percentage-based MBA allocation is enabled in the system") } // parse converts a string value into cacheAllocation type func (c CacheProportion) parse(minBits uint64) (cacheAllocation, error) { if c == "" { return nil, nil } if c[len(c)-1] == '%' { // Percentages of the max number of bits split := strings.SplitN(string(c)[0:len(c)-1], "-", 2) var allocation cacheAllocation if len(split) == 1 { pct, err := strconv.ParseUint(split[0], 10, 7) if err != nil { return allocation, err } if pct > 100 { return allocation, fmt.Errorf("invalid percentage value %q", c) } allocation = catPctAllocation(pct) } else { low, err := strconv.ParseUint(split[0], 10, 7) if err != nil { return allocation, err } high, err := strconv.ParseUint(split[1], 10, 7) if err != nil { return allocation, err } if low > high || low > 100 || high > 100 { return allocation, fmt.Errorf("invalid percentage range %q", c) } allocation = catPctRangeAllocation{lowPct: low, highPct: high} } return allocation, nil } // Absolute allocation var value uint64 var err error if strings.HasPrefix(string(c), "0x") { // Hex value value, err = strconv.ParseUint(string(c[2:]), 16, 64) if err != nil { return nil, err } } else { // Last, try "list" format (i.e. smthg like 0,2,5-9,...) tmp, err := listStrToBitmask(string(c)) value = uint64(tmp) if err != nil { return nil, err } } // Sanity check of absolute allocation: bitmask must (only) contain one // contiguous block of ones wide enough numOnes := bits.OnesCount64(value) if numOnes != bits.Len64(value)-bits.TrailingZeros64(value) { return nil, fmt.Errorf("invalid cache bitmask %q: more than one continuous block of ones", c) } if uint64(numOnes) < minBits { return nil, fmt.Errorf("invalid cache bitmask %q: number of bits less than %d", c, minBits) } return catAbsoluteAllocation(value), nil } golang-github-intel-goresctrl-0.3.0/pkg/rdt/info.go000066400000000000000000000211151432612301500222050ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdt import ( "bufio" "fmt" "io/ioutil" "os" "path/filepath" "sort" "strconv" "strings" ) // resctrlInfo contains information about the RDT support in the system type resctrlInfo struct { resctrlPath string resctrlMountOpts map[string]struct{} numClosids uint64 cat map[cacheLevel]catInfoAll l3mon l3MonInfo mb mbInfo } type cacheLevel string const ( L2 cacheLevel = "L2" L3 cacheLevel = "L3" ) type catInfoAll struct { cacheIds []uint64 unified catInfo code catInfo data catInfo } type catInfo struct { cbmMask bitmask minCbmBits uint64 shareableBits bitmask } type l3MonInfo struct { numRmids uint64 monFeatures []string } type mbInfo struct { cacheIds []uint64 bandwidthGran uint64 delayLinear uint64 minBandwidth uint64 mbpsEnabled bool // true if MBA_MBps is enabled } var mountInfoPath string = "/proc/mounts" // getInfo is a helper method for a "unified API" for getting L3 information func (i catInfoAll) getInfo() catInfo { switch { case i.code.Supported(): return i.code case i.data.Supported(): return i.data } return i.unified } func (i catInfoAll) cbmMask() bitmask { mask := i.getInfo().cbmMask if mask != 0 { return mask } return bitmask(^uint64(0)) } func (i catInfoAll) minCbmBits() uint64 { return i.getInfo().minCbmBits } func getRdtInfo() (*resctrlInfo, error) { var err error info := &resctrlInfo{cat: make(map[cacheLevel]catInfoAll)} info.resctrlPath, info.resctrlMountOpts, err = getResctrlMountInfo() if err != nil { return info, fmt.Errorf("failed to detect resctrl mount point: %v", err) } log.Infof("detected resctrl filesystem at %q", info.resctrlPath) // Check that RDT is available infopath := filepath.Join(info.resctrlPath, "info") if _, err := os.Stat(infopath); err != nil { return info, fmt.Errorf("failed to read RDT info from %q: %v", infopath, err) } // Check CAT feature available for _, cl := range []cacheLevel{L2, L3} { cat := catInfoAll{} catFeatures := map[string]*catInfo{ "": &cat.unified, "CODE": &cat.code, "DATA": &cat.data, } for suffix, i := range catFeatures { dir := string(cl) + suffix subpath := filepath.Join(infopath, dir) if _, err = os.Stat(subpath); err == nil { *i, info.numClosids, err = getCatInfo(subpath) if err != nil { return info, fmt.Errorf("failed to get %s info from %q: %v", dir, subpath, err) } } } if cat.getInfo().Supported() { cat.cacheIds, err = getCacheIds(info.resctrlPath, string(cl)) if err != nil { return info, fmt.Errorf("failed to get %s CAT cache IDs: %v", cl, err) } } info.cat[cl] = cat } // Check MON features available subpath := filepath.Join(infopath, "L3_MON") if _, err = os.Stat(subpath); err == nil { info.l3mon, err = getL3MonInfo(subpath) if err != nil { return info, fmt.Errorf("failed to get L3_MON info from %q: %v", subpath, err) } } // Check MBA feature available subpath = filepath.Join(infopath, "MB") if _, err = os.Stat(subpath); err == nil { info.mb, info.numClosids, err = getMBInfo(subpath) if err != nil { return info, fmt.Errorf("failed to get MBA info from %q: %v", subpath, err) } info.mb.cacheIds, err = getCacheIds(info.resctrlPath, "MB") if err != nil { return info, fmt.Errorf("failed to get MBA cache IDs: %v", err) } } return info, nil } func getCatInfo(basepath string) (catInfo, uint64, error) { var err error var numClosids uint64 info := catInfo{} info.cbmMask, err = readFileBitmask(filepath.Join(basepath, "cbm_mask")) if err != nil { return info, numClosids, err } info.minCbmBits, err = readFileUint64(filepath.Join(basepath, "min_cbm_bits")) if err != nil { return info, numClosids, err } info.shareableBits, err = readFileBitmask(filepath.Join(basepath, "shareable_bits")) if err != nil { return info, numClosids, err } numClosids, err = readFileUint64(filepath.Join(basepath, "num_closids")) if err != nil { return info, numClosids, err } return info, numClosids, nil } // Supported returns true if L3 cache allocation has is supported and enabled in the system func (i catInfo) Supported() bool { return i.cbmMask != 0 } func getL3MonInfo(basepath string) (l3MonInfo, error) { var err error info := l3MonInfo{} info.numRmids, err = readFileUint64(filepath.Join(basepath, "num_rmids")) if err != nil { return info, err } lines, err := readFileString(filepath.Join(basepath, "mon_features")) if err != nil { return info, err } info.monFeatures = strings.Split(lines, "\n") sort.Strings(info.monFeatures) return info, nil } // Supported returns true if L3 monitoring is supported and enabled in the system func (i l3MonInfo) Supported() bool { return i.numRmids != 0 && len(i.monFeatures) > 0 } func getMBInfo(basepath string) (mbInfo, uint64, error) { var err error var numClosids uint64 info := mbInfo{} info.bandwidthGran, err = readFileUint64(filepath.Join(basepath, "bandwidth_gran")) if err != nil { return info, numClosids, err } info.delayLinear, err = readFileUint64(filepath.Join(basepath, "delay_linear")) if err != nil { return info, numClosids, err } info.minBandwidth, err = readFileUint64(filepath.Join(basepath, "min_bandwidth")) if err != nil { return info, numClosids, err } numClosids, err = readFileUint64(filepath.Join(basepath, "num_closids")) if err != nil { return info, numClosids, err } // Detect MBps mode directly from mount options as it's not visible in MB // info directory _, mountOpts, err := getResctrlMountInfo() if err != nil { return info, numClosids, fmt.Errorf("failed to get resctrl mount options: %v", err) } if _, ok := mountOpts["mba_MBps"]; ok { info.mbpsEnabled = true } return info, numClosids, nil } // Supported returns true if memory bandwidth allocation has is supported and enabled in the system func (i mbInfo) Supported() bool { return i.minBandwidth != 0 } func getCacheIds(basepath string, prefix string) ([]uint64, error) { var ids []uint64 // Parse cache IDs from the root schemata data, err := readFileString(filepath.Join(basepath, "schemata")) if err != nil { return ids, fmt.Errorf("failed to read root schemata: %v", err) } for _, line := range strings.Split(data, "\n") { trimmed := strings.TrimSpace(line) lineSplit := strings.SplitN(trimmed, ":", 2) // Find line with given resource prefix if len(lineSplit) == 2 && strings.HasPrefix(lineSplit[0], prefix) { schema := strings.Split(lineSplit[1], ";") ids = make([]uint64, len(schema)) // Get individual cache configurations from the schema for idx, definition := range schema { split := strings.Split(definition, "=") if len(split) != 2 { return ids, fmt.Errorf("looks like an invalid schema %q", trimmed) } ids[idx], err = strconv.ParseUint(split[0], 10, 64) if err != nil { return ids, fmt.Errorf("failed to parse cache id in %q: %v", trimmed, err) } } return ids, nil } } return ids, fmt.Errorf("no %s resources in root schemata", prefix) } func getResctrlMountInfo() (string, map[string]struct{}, error) { mountOptions := map[string]struct{}{} f, err := os.Open(mountInfoPath) if err != nil { return "", mountOptions, err } defer f.Close() s := bufio.NewScanner(f) for s.Scan() { split := strings.Split(s.Text(), " ") if len(split) > 3 && split[2] == "resctrl" { opts := strings.Split(split[3], ",") for _, opt := range opts { mountOptions[opt] = struct{}{} } return split[1], mountOptions, nil } } return "", mountOptions, fmt.Errorf("resctrl not found in " + mountInfoPath) } func readFileUint64(path string) (uint64, error) { data, err := readFileString(path) if err != nil { return 0, err } return strconv.ParseUint(data, 10, 64) } func readFileBitmask(path string) (bitmask, error) { data, err := readFileString(path) if err != nil { return 0, err } value, err := strconv.ParseUint(data, 16, 64) return bitmask(value), err } func readFileString(path string) (string, error) { data, err := ioutil.ReadFile(path) return strings.TrimSpace(string(data)), err } golang-github-intel-goresctrl-0.3.0/pkg/rdt/kubernetes.go000066400000000000000000000055031432612301500234240ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdt import ( "fmt" "github.com/intel/goresctrl/pkg/kubernetes" ) const ( // RdtContainerAnnotation is the CRI level container annotation for setting // the RDT class (CLOS) of a container RdtContainerAnnotation = "io.kubernetes.cri.rdt-class" // RdtPodAnnotation is a Pod annotation for setting the RDT class (CLOS) of // all containers of the pod RdtPodAnnotation = "rdt.resources.beta.kubernetes.io/pod" // RdtPodAnnotationContainerPrefix is prefix for per-container Pod annotation // for setting the RDT class (CLOS) of one container of the pod RdtPodAnnotationContainerPrefix = "rdt.resources.beta.kubernetes.io/container." ) // ContainerClassFromAnnotations determines the effective RDT class of a // container from the Pod annotations and CRI level container annotations of a // container. Verifies that the class exists in goresctrl configuration and that // it is allowed to be used. func ContainerClassFromAnnotations(containerName string, containerAnnotations, podAnnotations map[string]string) (string, error) { clsName, clsOrigin := kubernetes.ContainerClassFromAnnotations( RdtContainerAnnotation, RdtPodAnnotation, RdtPodAnnotationContainerPrefix, containerName, containerAnnotations, podAnnotations) if clsOrigin != kubernetes.ClassOriginNotFound { if rdt == nil { return "", fmt.Errorf("RDT not initialized, class %q not available", clsName) } // Verify validity of class name if !IsQualifiedClassName(clsName) { return "", fmt.Errorf("unqualified RDT class name %q", clsName) } // If RDT has been initialized we check that the class exists if _, ok := rdt.getClass(clsName); !ok { return "", fmt.Errorf("RDT class %q does not exist in configuration", clsName) } // If classes have been configured by goresctrl if clsConf, ok := rdt.conf.Classes[unaliasClassName(clsName)]; ok { // Check that the class is allowed if clsOrigin == kubernetes.ClassOriginPodAnnotation && clsConf.Kubernetes.DenyPodAnnotation { return "", fmt.Errorf("RDT class %q not allowed from Pod annotations", clsName) } else if clsOrigin == kubernetes.ClassOriginContainerAnnotation && clsConf.Kubernetes.DenyContainerAnnotation { return "", fmt.Errorf("RDT class %q not allowed from Container annotation", clsName) } } } return clsName, nil } golang-github-intel-goresctrl-0.3.0/pkg/rdt/kubernetes_test.go000066400000000000000000000062241432612301500244640ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdt import ( "testing" ) func TestContainerClassFromAnnotations(t *testing.T) { mockRdt := &control{ classes: make(map[string]*ctrlGroup), } containerName := "test-container" containerAnnotations := map[string]string{} podAnnotations := map[string]string{} // Helper function for checking test cases tc := func(expectError bool, expectedClsName string) { cls, err := ContainerClassFromAnnotations(containerName, containerAnnotations, podAnnotations) if !expectError && err != nil { t.Errorf("unexpected error: %v", err) } else if expectError && err == nil { t.Errorf("unexpected success setting RDT class to %q", cls) } else if cls != expectedClsName { t.Errorf("invalid rdt class, expecting %q, got %q", expectedClsName, cls) } } // // 1. Test container annotation // // Should succeed when rdt is uninitialized but annotations are empty rdt = nil tc(false, "") // Should fail when rdt is uninitialized but annotations point to a class containerAnnotations = map[string]string{RdtContainerAnnotation: "class-1"} podAnnotations = map[string]string{ RdtPodAnnotationContainerPrefix + containerName: "class-2", RdtPodAnnotation: "class-3"} tc(true, "") // Mock configured rdt which enables the functionality rdt = mockRdt // Should fail with an empty set of classes tc(true, "") // Should succeed when the class exists but no configuration has been set ("discovery mode") mockRdt.classes = map[string]*ctrlGroup{"": nil, "class-1": nil, "class-2": nil, "class-3": nil} tc(false, "class-1") // Should succeed with default class config mockRdt.conf.Classes = classSet{"class-1": &classConfig{}, "class-2": &classConfig{}, "class-3": &classConfig{}} tc(false, "class-1") // Should fail when container annotation has been denied in class ocnfig mockRdt.conf.Classes["class-1"].Kubernetes.DenyContainerAnnotation = true tc(true, "") // Test invalid class name containerAnnotations[RdtContainerAnnotation] = "foo/bar" tc(true, "") // // 2. Test per-container Pod annotation // delete(containerAnnotations, RdtContainerAnnotation) tc(false, "class-2") // Should fail when pod annotations for the class are denied mockRdt.conf.Classes["class-2"].Kubernetes.DenyPodAnnotation = true tc(true, "") // // 3. Test pod-wide Pod annotation // delete(podAnnotations, RdtPodAnnotationContainerPrefix+containerName) tc(false, "class-3") // Should fail when pod annotations for the class are denied mockRdt.conf.Classes["class-3"].Kubernetes.DenyPodAnnotation = true tc(true, "") // // Test empty annotations // delete(podAnnotations, RdtPodAnnotation) tc(false, "") } golang-github-intel-goresctrl-0.3.0/pkg/rdt/prometheus.go000066400000000000000000000061671432612301500234570ustar00rootroot00000000000000/* Copyright 2020 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdt import ( "fmt" "sync" "github.com/prometheus/client_golang/prometheus" ) var customLabels []string = []string{} // collector implements prometheus.Collector interface type collector struct { descriptors map[string]*prometheus.Desc } // NewCollector creates new Prometheus collector of RDT metrics func NewCollector() (prometheus.Collector, error) { c := &collector{descriptors: make(map[string]*prometheus.Desc)} return c, nil } // RegisterCustomPrometheusLabels registers monitor group annotations to be // exported as Prometheus metrics labels func RegisterCustomPrometheusLabels(names ...string) { Names: for _, n := range names { for _, c := range customLabels { if n == c { break Names } } customLabels = append(customLabels, n) } } // Describe method of the prometheus.Collector interface func (c *collector) Describe(ch chan<- *prometheus.Desc) { for resource, features := range GetMonFeatures() { switch resource { case MonResourceL3: for _, f := range features { ch <- c.describeL3(f) } } } } // Collect method of the prometheus.Collector interface func (c collector) Collect(ch chan<- prometheus.Metric) { var wg sync.WaitGroup for _, cls := range GetClasses() { for _, monGrp := range cls.GetMonGroups() { wg.Add(1) g := monGrp go func() { defer wg.Done() c.collectGroupMetrics(ch, g) }() } } wg.Wait() } func (c *collector) describeL3(feature string) *prometheus.Desc { d, ok := c.descriptors[feature] if !ok { name := "l3_" + feature help := "L3 " + feature switch feature { case "llc_occupancy": help = "L3 (LLC) occupancy" case "mbm_local_bytes": help = "bytes transferred to/from local memory through LLC" case "mbm_total_bytes": help = "total bytes transferred to/from memory through LLC" } labels := append([]string{"rdt_class", "rdt_mon_group", "cache_id"}, customLabels...) d = prometheus.NewDesc(name, help, labels, nil) c.descriptors[feature] = d } return d } func (c *collector) collectGroupMetrics(ch chan<- prometheus.Metric, mg MonGroup) { allData := mg.GetMonData() annotations := mg.GetAnnotations() customLabelValues := make([]string, len(customLabels)) for i, name := range customLabels { customLabelValues[i] = annotations[name] } for cacheID, data := range allData.L3 { for feature, value := range data { labels := append([]string{mg.Parent().Name(), mg.Name(), fmt.Sprint(cacheID)}, customLabelValues...) ch <- prometheus.MustNewConstMetric( c.describeL3(feature), prometheus.CounterValue, float64(value), labels..., ) } } } golang-github-intel-goresctrl-0.3.0/pkg/rdt/rdt.go000066400000000000000000000535211432612301500220510ustar00rootroot00000000000000/* Copyright 2019 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // Package rdt implements an API for managing Intel® RDT technologies via the // resctrl pseudo-filesystem of the Linux kernel. It provides flexible // configuration with a hierarchical approach for easy management of exclusive // cache allocations. // // Goresctrl supports all available RDT technologies, i.e. L2 and L3 Cache // Allocation (CAT) with Code and Data Prioritization (CDP) and Memory // Bandwidth Allocation (MBA) plus Cache Monitoring (CMT) and Memory Bandwidth // Monitoring (MBM). // // Basic usage example: // rdt.SetLogger(logrus.New()) // // if err := rdt.Initialize(""); err != nil { // return fmt.Errorf("RDT not supported: %v", err) // } // // if err := rdt.SetConfigFromFile("/path/to/rdt.conf.yaml", false); err != nil { // return fmt.Errorf("RDT configuration failed: %v", err) // } // // if cls, ok := rdt.GetClass("my-class"); ok { // // Set PIDs 12345 and 12346 to class "my-class" // if err := cls.AddPids("12345", "12346"); err != nil { // return fmt.Errorf("failed to add PIDs to RDT class: %v", err) // } // } package rdt import ( "errors" "fmt" "io/ioutil" stdlog "log" "os" "path/filepath" "sort" "strconv" "strings" "syscall" "sigs.k8s.io/yaml" grclog "github.com/intel/goresctrl/pkg/log" "github.com/intel/goresctrl/pkg/utils" ) const ( // RootClassName is the name we use in our config for the special class // that configures the "root" resctrl group of the system RootClassName = "system/default" // RootClassAlias is an alternative name for the root class RootClassAlias = "" ) type control struct { grclog.Logger resctrlGroupPrefix string conf config rawConf Config classes map[string]*ctrlGroup } var log grclog.Logger = grclog.NewLoggerWrapper(stdlog.New(os.Stderr, "[ rdt ] ", 0)) var info *resctrlInfo var rdt *control // Function for removing resctrl groups from the filesystem. This is // configurable because of unit tests. var groupRemoveFunc func(string) error = os.Remove // CtrlGroup defines the interface of one goresctrl managed RDT class. It maps // to one CTRL group directory in the goresctrl pseudo-filesystem. type CtrlGroup interface { ResctrlGroup // CreateMonGroup creates a new monitoring group under this CtrlGroup. CreateMonGroup(name string, annotations map[string]string) (MonGroup, error) // DeleteMonGroup deletes a monitoring group from this CtrlGroup. DeleteMonGroup(name string) error // DeleteMonGroups deletes all monitoring groups from this CtrlGroup. DeleteMonGroups() error // GetMonGroup returns a specific monitoring group under this CtrlGroup. GetMonGroup(name string) (MonGroup, bool) // GetMonGroups returns all monitoring groups under this CtrlGroup. GetMonGroups() []MonGroup } // ResctrlGroup is the generic interface for resctrl CTRL and MON groups. It // maps to one CTRL or MON group directory in the goresctrl pseudo-filesystem. type ResctrlGroup interface { // Name returns the name of the group. Name() string // GetPids returns the process ids assigned to the group. GetPids() ([]string, error) // AddPids assigns the given process ids to the group. AddPids(pids ...string) error // GetMonData retrieves the monitoring data of the group. GetMonData() MonData } // MonGroup represents the interface to a RDT monitoring group. It maps to one // MON group in the goresctrl filesystem. type MonGroup interface { ResctrlGroup // Parent returns the CtrlGroup under which the monitoring group exists. Parent() CtrlGroup // GetAnnotations returns the annotations stored to the monitoring group. GetAnnotations() map[string]string } // MonData contains monitoring stats of one monitoring group. type MonData struct { L3 MonL3Data } // MonL3Data contains L3 monitoring stats of one monitoring group. type MonL3Data map[uint64]MonLeafData // MonLeafData represents the raw numerical stats from one RDT monitor data leaf. type MonLeafData map[string]uint64 // MonResource is the type of RDT monitoring resource. type MonResource string const ( // MonResourceL3 is the RDT L3 cache monitor resource. MonResourceL3 MonResource = "l3" ) type ctrlGroup struct { resctrlGroup monPrefix string monGroups map[string]*monGroup } type monGroup struct { resctrlGroup annotations map[string]string } type resctrlGroup struct { prefix string name string parent *ctrlGroup // parent for MON groups } // SetLogger sets the logger instance to be used by the package. This function // may be called even before Initialize(). func SetLogger(l grclog.Logger) { log = l if rdt != nil { rdt.setLogger(l) } } // Initialize detects RDT from the system and initializes control interface of // the package. func Initialize(resctrlGroupPrefix string) error { var err error info = nil rdt = nil // Get info from the resctrl filesystem info, err = getRdtInfo() if err != nil { return err } r := &control{Logger: log, resctrlGroupPrefix: resctrlGroupPrefix} // NOTE: we lose monitoring group annotations (i.e. prometheus metrics // labels) on re-init if r.classes, err = r.classesFromResctrlFs(); err != nil { return fmt.Errorf("failed to initialize classes from resctrl fs: %v", err) } rdt = r return nil } // DiscoverClasses discovers existing classes from the resctrl filesystem. // Makes it possible to discover gropus with another prefix than was set with // Initialize(). The original prefix is still used for monitoring groups. func DiscoverClasses(resctrlGroupPrefix string) error { if rdt != nil { return rdt.discoverFromResctrl(resctrlGroupPrefix) } return fmt.Errorf("rdt not initialized") } // SetConfig (re-)configures the resctrl filesystem according to the specified // configuration. func SetConfig(c *Config, force bool) error { if rdt != nil { return rdt.setConfig(c, force) } return fmt.Errorf("rdt not initialized") } // SetConfigFromData takes configuration as raw data, parses it and // reconfigures the resctrl filesystem. func SetConfigFromData(data []byte, force bool) error { cfg := &Config{} if err := yaml.Unmarshal(data, &cfg); err != nil { return fmt.Errorf("failed to parse configuration data: %v", err) } return SetConfig(cfg, force) } // SetConfigFromFile reads configuration from the filesystem and reconfigures // the resctrl filesystem. func SetConfigFromFile(path string, force bool) error { data, err := ioutil.ReadFile(path) if err != nil { return fmt.Errorf("failed to read config file: %v", err) } if err := SetConfigFromData(data, force); err != nil { return err } log.Infof("configuration successfully loaded from %q", path) return nil } // GetClass returns one RDT class. func GetClass(name string) (CtrlGroup, bool) { if rdt != nil { return rdt.getClass(name) } return nil, false } // GetClasses returns all available RDT classes. func GetClasses() []CtrlGroup { if rdt != nil { return rdt.getClasses() } return []CtrlGroup{} } // MonSupported returns true if RDT monitoring features are available. func MonSupported() bool { if rdt != nil { return rdt.monSupported() } return false } // GetMonFeatures returns the available monitoring stats of each available // monitoring technology. func GetMonFeatures() map[MonResource][]string { if rdt != nil { return rdt.getMonFeatures() } return map[MonResource][]string{} } // IsQualifiedClassName returns true if given string qualifies as a class name func IsQualifiedClassName(name string) bool { // Must be qualified as a file name return name == RootClassName || (len(name) < 4096 && name != "." && name != ".." && !strings.ContainsAny(name, "/\n")) } func (c *control) getClass(name string) (CtrlGroup, bool) { cls, ok := c.classes[unaliasClassName(name)] return cls, ok } func (c *control) getClasses() []CtrlGroup { ret := make([]CtrlGroup, 0, len(c.classes)) for _, v := range c.classes { ret = append(ret, v) } sort.Slice(ret, func(i, j int) bool { return ret[i].Name() < ret[j].Name() }) return ret } func (c *control) monSupported() bool { return info.l3mon.Supported() } func (c *control) getMonFeatures() map[MonResource][]string { ret := make(map[MonResource][]string) if info.l3mon.Supported() { ret[MonResourceL3] = append([]string{}, info.l3mon.monFeatures...) } return ret } func (c *control) setLogger(l grclog.Logger) { c.Logger = l } func (c *control) setConfig(newConfig *Config, force bool) error { c.Infof("configuration update") conf, err := (*newConfig).resolve() if err != nil { return fmt.Errorf("invalid configuration: %v", err) } err = c.configureResctrl(conf, force) if err != nil { return fmt.Errorf("resctrl configuration failed: %v", err) } c.conf = conf // TODO: we'd better create a deep copy c.rawConf = *newConfig c.Infof("configuration finished") return nil } func (c *control) configureResctrl(conf config, force bool) error { grclog.DebugBlock(c, "applying resolved config:", " ", "%s", utils.DumpJSON(conf)) // Remove stale resctrl groups classesFromFs, err := c.classesFromResctrlFs() if err != nil { return err } for name, cls := range classesFromFs { if _, ok := conf.Classes[cls.name]; !isRootClass(cls.name) && !ok { if !force { tasks, err := cls.GetPids() if err != nil { return fmt.Errorf("failed to get resctrl group tasks: %v", err) } if len(tasks) > 0 { return fmt.Errorf("refusing to remove non-empty resctrl group %q", cls.relPath("")) } } log.Debugf("removing existing resctrl group %q", cls.relPath("")) err = groupRemoveFunc(cls.path("")) if err != nil { return fmt.Errorf("failed to remove resctrl group %q: %v", cls.relPath(""), err) } delete(c.classes, name) } } for name, cls := range c.classes { if _, ok := conf.Classes[cls.name]; !ok || cls.prefix != c.resctrlGroupPrefix { if !isRootClass(cls.name) { log.Debugf("dropping stale class %q (%q)", name, cls.path("")) delete(c.classes, name) } } } if _, ok := c.classes[RootClassName]; !ok { log.Warnf("root class missing from runtime data, re-adding...") c.classes[RootClassName] = classesFromFs[RootClassName] } // Try to apply given configuration for name, class := range conf.Classes { if _, ok := c.classes[name]; !ok { cg, err := newCtrlGroup(c.resctrlGroupPrefix, c.resctrlGroupPrefix, name) if err != nil { return err } c.classes[name] = cg } partition := conf.Partitions[class.Partition] if err := c.classes[name].configure(name, class, partition, conf.Options); err != nil { return err } } if err := c.pruneMonGroups(); err != nil { return err } return nil } func (c *control) discoverFromResctrl(prefix string) error { c.Debugf("running class discovery from resctrl filesystem using prefix %q", prefix) classesFromFs, err := c.classesFromResctrlFsPrefix(prefix) if err != nil { return err } // Drop stale classes for name, cls := range c.classes { if _, ok := classesFromFs[cls.name]; !ok || cls.prefix != prefix { if !isRootClass(cls.name) { log.Debugf("dropping stale class %q (%q)", name, cls.path("")) delete(c.classes, name) } } } for name, cls := range classesFromFs { if _, ok := c.classes[name]; !ok { c.classes[name] = cls log.Debugf("adding discovered class %q (%q)", name, cls.path("")) } } if err := c.pruneMonGroups(); err != nil { return err } return nil } func (c *control) classesFromResctrlFs() (map[string]*ctrlGroup, error) { return c.classesFromResctrlFsPrefix(c.resctrlGroupPrefix) } func (c *control) classesFromResctrlFsPrefix(prefix string) (map[string]*ctrlGroup, error) { names := []string{RootClassName} if g, err := resctrlGroupsFromFs(prefix, info.resctrlPath); err != nil { return nil, err } else { for _, n := range g { if prefix != c.resctrlGroupPrefix && strings.HasPrefix(n, c.resctrlGroupPrefix) && strings.HasPrefix(c.resctrlGroupPrefix, prefix) { // Skip groups in the standard namespace continue } names = append(names, n[len(prefix):]) } } classes := make(map[string]*ctrlGroup, len(names)+1) for _, name := range names { g, err := newCtrlGroup(prefix, c.resctrlGroupPrefix, name) if err != nil { return nil, err } classes[name] = g } return classes, nil } func (c *control) pruneMonGroups() error { for name, cls := range c.classes { if err := cls.pruneMonGroups(); err != nil { return fmt.Errorf("failed to prune stale monitoring groups of %q: %v", name, err) } } return nil } func (c *control) readRdtFile(rdtPath string) ([]byte, error) { return ioutil.ReadFile(filepath.Join(info.resctrlPath, rdtPath)) } func (c *control) writeRdtFile(rdtPath string, data []byte) error { if err := ioutil.WriteFile(filepath.Join(info.resctrlPath, rdtPath), data, 0644); err != nil { return c.cmdError(err) } return nil } func (c *control) cmdError(origErr error) error { errData, readErr := c.readRdtFile(filepath.Join("info", "last_cmd_status")) if readErr != nil { return origErr } cmdStatus := strings.TrimSpace(string(errData)) if len(cmdStatus) > 0 && cmdStatus != "ok" { return fmt.Errorf("%s", cmdStatus) } return origErr } func newCtrlGroup(prefix, monPrefix, name string) (*ctrlGroup, error) { cg := &ctrlGroup{ resctrlGroup: resctrlGroup{prefix: prefix, name: name}, monPrefix: monPrefix, } if err := os.Mkdir(cg.path(""), 0755); err != nil && !os.IsExist(err) { return nil, err } var err error cg.monGroups, err = cg.monGroupsFromResctrlFs() if err != nil { return nil, fmt.Errorf("error when retrieving existing monitor groups: %v", err) } return cg, nil } func (c *ctrlGroup) CreateMonGroup(name string, annotations map[string]string) (MonGroup, error) { if mg, ok := c.monGroups[name]; ok { return mg, nil } log.Debugf("creating monitoring group %s/%s", c.name, name) mg, err := newMonGroup(c.monPrefix, name, c, annotations) if err != nil { return nil, fmt.Errorf("failed to create new monitoring group %q: %v", name, err) } c.monGroups[name] = mg return mg, err } func (c *ctrlGroup) DeleteMonGroup(name string) error { mg, ok := c.monGroups[name] if !ok { log.Warnf("trying to delete non-existent mon group %s/%s", c.name, name) return nil } log.Debugf("deleting monitoring group %s/%s", c.name, name) if err := groupRemoveFunc(mg.path("")); err != nil { return fmt.Errorf("failed to remove monitoring group %q: %v", mg.relPath(""), err) } delete(c.monGroups, name) return nil } func (c *ctrlGroup) DeleteMonGroups() error { for name := range c.monGroups { if err := c.DeleteMonGroup(name); err != nil { return err } } return nil } func (c *ctrlGroup) GetMonGroup(name string) (MonGroup, bool) { mg, ok := c.monGroups[name] return mg, ok } func (c *ctrlGroup) GetMonGroups() []MonGroup { ret := make([]MonGroup, 0, len(c.monGroups)) for _, v := range c.monGroups { ret = append(ret, v) } sort.Slice(ret, func(i, j int) bool { return ret[i].Name() < ret[j].Name() }) return ret } func (c *ctrlGroup) configure(name string, class *classConfig, partition *partitionConfig, options Options) error { schemata := "" // Handle cache allocation for _, lvl := range []cacheLevel{L2, L3} { switch { case info.cat[lvl].unified.Supported(): schema, err := class.CATSchema[lvl].toStr(catSchemaTypeUnified, partition.CAT[lvl]) if err != nil { return err } schemata += schema case info.cat[lvl].data.Supported() || info.cat[lvl].code.Supported(): schema, err := class.CATSchema[lvl].toStr(catSchemaTypeCode, partition.CAT[lvl]) if err != nil { return err } schemata += schema schema, err = class.CATSchema[lvl].toStr(catSchemaTypeData, partition.CAT[lvl]) if err != nil { return err } schemata += schema default: if class.CATSchema[lvl].Alloc != nil && !options.cat(lvl).Optional { return fmt.Errorf("%s cache allocation for %q specified in configuration but not supported by system", lvl, name) } } } // Handle memory bandwidth allocation switch { case info.mb.Supported(): schemata += class.MBSchema.toStr(partition.MB) default: if class.MBSchema != nil && !options.MB.Optional { return fmt.Errorf("memory bandwidth allocation for %q specified in configuration but not supported by system", name) } } if len(schemata) > 0 { log.Debugf("writing schemata %q to %q", schemata, c.relPath("")) if err := rdt.writeRdtFile(c.relPath("schemata"), []byte(schemata)); err != nil { return err } } else { log.Debugf("empty schemata") } return nil } func (c *ctrlGroup) monGroupsFromResctrlFs() (map[string]*monGroup, error) { names, err := resctrlGroupsFromFs(c.monPrefix, c.path("mon_groups")) if err != nil && !os.IsNotExist(err) { return nil, err } grps := make(map[string]*monGroup, len(names)) for _, name := range names { name = name[len(c.monPrefix):] mg, err := newMonGroup(c.monPrefix, name, c, nil) if err != nil { return nil, err } grps[name] = mg } return grps, nil } // Remove empty monitoring groups func (c *ctrlGroup) pruneMonGroups() error { for name, mg := range c.monGroups { pids, err := mg.GetPids() if err != nil { return fmt.Errorf("failed to get pids for monitoring group %q: %v", mg.relPath(""), err) } if len(pids) == 0 { if err := c.DeleteMonGroup(name); err != nil { return fmt.Errorf("failed to remove monitoring group %q: %v", mg.relPath(""), err) } } } return nil } func (r *resctrlGroup) Name() string { return r.name } func (r *resctrlGroup) GetPids() ([]string, error) { data, err := rdt.readRdtFile(r.relPath("tasks")) if err != nil { return []string{}, err } split := strings.Split(strings.TrimSpace(string(data)), "\n") if len(split[0]) > 0 { return split, nil } return []string{}, nil } func (r *resctrlGroup) AddPids(pids ...string) error { f, err := os.OpenFile(r.path("tasks"), os.O_WRONLY, 0644) if err != nil { return err } defer f.Close() for _, pid := range pids { if _, err := f.WriteString(pid + "\n"); err != nil { if errors.Is(err, syscall.ESRCH) { log.Debugf("no task %s", pid) } else { return fmt.Errorf("failed to assign processes %v to class %q: %v", pids, r.name, rdt.cmdError(err)) } } } return nil } func (r *resctrlGroup) GetMonData() MonData { m := MonData{} if info.l3mon.Supported() { l3, err := r.getMonL3Data() if err != nil { log.Warnf("failed to retrieve L3 monitoring data: %v", err) } else { m.L3 = l3 } } return m } func (r *resctrlGroup) getMonL3Data() (MonL3Data, error) { files, err := ioutil.ReadDir(r.path("mon_data")) if err != nil { return nil, err } m := MonL3Data{} for _, file := range files { name := file.Name() if strings.HasPrefix(name, "mon_L3_") { // Parse cache id from the dirname id, err := strconv.ParseUint(strings.TrimPrefix(name, "mon_L3_"), 10, 32) if err != nil { // Just print a warning, we try to retrieve as much info as possible log.Warnf("error parsing L3 monitor data directory name %q: %v", name, err) continue } data, err := r.getMonLeafData(filepath.Join("mon_data", name)) if err != nil { log.Warnf("failed to read monitor data: %v", err) continue } m[id] = data } } return m, nil } func (r *resctrlGroup) getMonLeafData(path string) (MonLeafData, error) { files, err := ioutil.ReadDir(r.path(path)) if err != nil { return nil, err } m := make(MonLeafData, len(files)) for _, file := range files { name := file.Name() // We expect that all the files in the dir are regular files val, err := readFileUint64(r.path(path, name)) if err != nil { // Just print a warning, we want to retrieve as much info as possible log.Warnf("error reading data file: %v", err) continue } m[name] = val } return m, nil } func (r *resctrlGroup) relPath(elem ...string) string { if r.parent == nil { if r.name == RootClassName { return filepath.Join(elem...) } return filepath.Join(append([]string{r.prefix + r.name}, elem...)...) } // Parent is only intended for MON groups - non-root CTRL groups are considered // as peers to the root CTRL group (as they are in HW) and do not have a parent return r.parent.relPath(append([]string{"mon_groups", r.prefix + r.name}, elem...)...) } func (r *resctrlGroup) path(elem ...string) string { return filepath.Join(info.resctrlPath, r.relPath(elem...)) } func newMonGroup(prefix string, name string, parent *ctrlGroup, annotations map[string]string) (*monGroup, error) { mg := &monGroup{ resctrlGroup: resctrlGroup{prefix: prefix, name: name, parent: parent}, annotations: make(map[string]string, len(annotations))} if err := os.Mkdir(mg.path(""), 0755); err != nil && !os.IsExist(err) { return nil, err } for k, v := range annotations { mg.annotations[k] = v } return mg, nil } func (m *monGroup) Parent() CtrlGroup { return m.parent } func (m *monGroup) GetAnnotations() map[string]string { a := make(map[string]string, len(m.annotations)) for k, v := range m.annotations { a[k] = v } return a } func resctrlGroupsFromFs(prefix string, path string) ([]string, error) { files, err := ioutil.ReadDir(path) if err != nil { return nil, err } grps := make([]string, 0, len(files)) for _, file := range files { filename := file.Name() if strings.HasPrefix(filename, prefix) { if s, err := os.Stat(filepath.Join(path, filename, "tasks")); err == nil && !s.IsDir() { grps = append(grps, filename) } } } return grps, nil } func isRootClass(name string) bool { return name == RootClassName || name == RootClassAlias } func unaliasClassName(name string) string { if isRootClass(name) { return RootClassName } return name } golang-github-intel-goresctrl-0.3.0/pkg/rdt/rdt_test.go000066400000000000000000001302451432612301500231070ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdt import ( "io/ioutil" stdlog "log" "os" "os/exec" "path/filepath" "regexp" "sort" "strings" "testing" "sigs.k8s.io/yaml" "github.com/google/go-cmp/cmp" grclog "github.com/intel/goresctrl/pkg/log" "github.com/intel/goresctrl/pkg/testutils" "github.com/intel/goresctrl/pkg/utils" testdata "github.com/intel/goresctrl/test/data" ) const mockGroupPrefix string = "goresctrl." type mockResctrlFs struct { t *testing.T origDir string baseDir string } func newMockResctrlFs(t *testing.T, name, mountOpts string) (*mockResctrlFs, error) { var err error m := &mockResctrlFs{t: t} m.origDir = testdata.Path(name) m.baseDir, err = ioutil.TempDir("", "goresctrl.test.") if err != nil { return nil, err } // Create resctrl filesystem mock m.copyFromOrig("", "") // Create mountinfo mock mountInfoPath = filepath.Join(m.baseDir, "mounts") resctrlPath := filepath.Join(m.baseDir, "resctrl") data := "resctrl " + resctrlPath + " resctrl " + mountOpts + " 0 0\n" if err := ioutil.WriteFile(mountInfoPath, []byte(data), 0644); err != nil { m.delete() return nil, err } return m, nil } func (m *mockResctrlFs) delete() { if err := os.RemoveAll(m.baseDir); err != nil { m.t.Fatalf("failed to delete mock resctrl fs: %v", err) } } func (m *mockResctrlFs) initMockMonGroup(class, name string) { m.copyFromOrig(filepath.Join("mon_groups", "example"), filepath.Join(mockGroupPrefix+class, "mon_groups", mockGroupPrefix+name)) } func (m *mockResctrlFs) copyFromOrig(relSrc, relDst string) { absSrc := filepath.Join(m.origDir, relSrc) if s, err := os.Stat(absSrc); err != nil { m.t.Fatalf("%v", err) } else if s.IsDir() { absSrc = filepath.Join(absSrc, ".") } absDst := filepath.Join(m.baseDir, "resctrl", relDst) cmd := exec.Command("cp", "-r", absSrc, absDst) if err := cmd.Run(); err != nil { m.t.Fatalf("failed to copy mock data %q -> %q: %v", absSrc, absDst, err) } } func (m *mockResctrlFs) verifyTextFile(relPath, content string) { verifyTextFile(m.t, filepath.Join(m.baseDir, "resctrl", relPath), content) } func verifyTextFile(t *testing.T, path, content string) { data, err := ioutil.ReadFile(path) if err != nil { t.Fatalf("failed to read %q: %v", path, err) } if string(data) != content { t.Fatalf("unexpected content in %q\nexpected:\n %q\nfound:\n %q", path, content, data) } } func parseTestConfig(t *testing.T, data string) *Config { c := &Config{} if err := yaml.Unmarshal([]byte(data), c); err != nil { t.Fatalf("failed to parse rdt config: %v", err) } return c } // TestRdt tests the rdt public API, i.e. exported functionality of the package func TestRdt(t *testing.T) { const rdtTestConfig string = ` partitions: priority: l3Allocation: all: 60% mbAllocation: all: [100%] classes: Guaranteed: l3Allocation: all: 100% default: l3Allocation: all: 40% mbAllocation: all: [100%] classes: Burstable: l3Allocation: all: 100% mbAllocation: all: [66%] BestEffort: l3Allocation: all: 66% mbAllocation: all: [33%] kubernetes: denyPodAnnotation: true kubernetes: allowedPodAnnotationClasses: [bar, foo] ` verifyGroupNames := func(a interface{}, b []string) { var names []string switch v := a.(type) { case []CtrlGroup: for _, g := range v { names = append(names, g.Name()) } case []MonGroup: for _, g := range v { names = append(names, g.Name()) } default: t.Errorf("Invalid type '%T' in verifyGroupNames()", a) return } if len(b) == 0 && len(names) == 0 { return } sort.Strings(names) sort.Strings(b) if !cmp.Equal(names, b) { t.Errorf("unexpected class/group names: expected %s got %s", b, names) } } // Set group remove function so that mock groups can be removed groupRemoveFunc = os.RemoveAll // // 1. test uninitialized interface // rdt = nil SetLogger(grclog.NewLoggerWrapper(stdlog.New(os.Stderr, "[ rdt-test-1 ] ", 0))) if err := SetConfig(&Config{}, false); err == nil { t.Errorf("setting config on uninitialized rdt succeeded unexpectedly") } if classes := GetClasses(); len(classes) != 0 { t.Errorf("uninitialized rdt contains classes %s", classes) } if _, ok := GetClass(""); ok { t.Errorf("expected to not get a class with empty name") } if MonSupported() { t.Errorf("unitialized rdt claims monitoring to be supported") } if features := GetMonFeatures(); len(features) != 0 { t.Errorf("uninitialized rdt returned monitoring features %s", features) } // // 2. Test setting up RDT with L3 L3_MON and MB support // mockFs, err := newMockResctrlFs(t, "resctrl.full", "") if err != nil { t.Fatalf("failed to set up mock resctrl fs: %v", err) } defer mockFs.delete() if err := Initialize(mockGroupPrefix); err != nil { t.Fatalf("rdt initialization failed: %v", err) } // Check that existing groups were read correctly on init classes := GetClasses() verifyGroupNames(classes, []string{"Guaranteed", "Stale", RootClassName}) cls, _ := GetClass(RootClassName) verifyGroupNames(cls.GetMonGroups(), []string{}) cls, _ = GetClass("Guaranteed") verifyGroupNames(cls.GetMonGroups(), []string{"predefined_group_empty", "predefined_group_live"}) cls, _ = GetClass("Stale") if err := cls.AddPids("99"); err != nil { t.Fatalf("AddPids() failed: %v", err) } // Invalid test config content should cause an error if err := SetConfigFromData([]byte("partitions: foo"), true); err == nil { t.Fatalf("rdt configuration with invalid file succeeded unexpetedly") } // Non-existent configuration file should cause an error if err := SetConfigFromFile("non-existent-config-file", true); err == nil { t.Fatalf("rdt configuration with non-existent file succeeded unexpetedly") } // Configuration should fail as "Stale" class has pids assigned to it testConfigFile := testutils.CreateTempFile(t, rdtTestConfig) defer os.Remove(testConfigFile) if err := SetConfigFromFile(testConfigFile, false); err == nil { t.Fatalf("rdt configuration succeeded unexpetedly") } // Forced configuration should succeed if err := SetConfigFromFile(testConfigFile, true); err != nil { t.Fatalf("rdt forced configuration failed: %v", err) } // Check that KubernetesOptions of classes are parsed and propagated correctly if !rdt.conf.Classes["BestEffort"].Kubernetes.DenyPodAnnotation { t.Fatal("DenyPodAnnotation of class BestEffort should be 'true'") } // Empty mon group(s) should be pruned after configuration cls, _ = GetClass("Guaranteed") verifyGroupNames(cls.GetMonGroups(), []string{"predefined_group_live"}) // Check that SetLogger() takes effect in the control interface, too l := grclog.NewLoggerWrapper(stdlog.New(os.Stderr, "[ rdt-test-2 ] ", 0)) SetLogger(l) if l != rdt.Logger { t.Errorf("unexpected logger implementation") } // Check that the path() and relPath() methods work correctly if p := rdt.classes["Guaranteed"].path("foo"); p != filepath.Join(mockFs.baseDir, "resctrl", "goresctrl.Guaranteed", "foo") { t.Errorf("path() returned wrong path %q", p) } if p := rdt.classes["Guaranteed"].relPath("foo"); p != filepath.Join("goresctrl.Guaranteed", "foo") { t.Errorf("relPath() returned wrong path %q", p) } // Verify that ctrl groups are correctly configured mockFs.verifyTextFile(rdt.classes["BestEffort"].relPath("schemata"), "L3:0=3f;1=3f;2=3f;3=3f\nMB:0=33;1=33;2=33;3=33\n") mockFs.verifyTextFile(rdt.classes["Burstable"].relPath("schemata"), "L3:0=ff;1=ff;2=ff;3=ff\nMB:0=66;1=66;2=66;3=66\n") mockFs.verifyTextFile(rdt.classes["Guaranteed"].relPath("schemata"), "L3:0=fff00;1=fff00;2=fff00;3=fff00\nMB:0=100;1=100;2=100;3=100\n") // Verify that existing goresctrl monitor groups were removed for _, cls := range []string{RootClassName, "Guaranteed"} { files, _ := ioutil.ReadDir(rdt.classes[cls].path("mon_groups")) for _, f := range files { if strings.HasPrefix(mockGroupPrefix, f.Name()) { t.Errorf("unexpected monitor group found %q", f.Name()) } } } // Verify GetClasses classes = GetClasses() verifyGroupNames(classes, []string{"BestEffort", "Burstable", "Guaranteed", RootClassName}) // Verify assigning pids to classes (ctrl groups) cls, _ = GetClass("Guaranteed") if n := cls.Name(); n != "Guaranteed" { t.Errorf("CtrlGroup.Name() returned %q, expected %q", n, "Guaranteed") } pids := []string{"10", "11", "12"} if err := cls.AddPids(pids...); err != nil { t.Errorf("AddPids() failed: %v", err) } if p, err := cls.GetPids(); err != nil { t.Errorf("GetPids() failed: %v", err) } else if !cmp.Equal(p, pids) { t.Errorf("GetPids() returned %s, expected %s", p, pids) } mockFs.verifyTextFile(rdt.classes["Guaranteed"].relPath("tasks"), "10\n11\n12\n") // Verify MonSupported and GetMonFeatures if !MonSupported() { t.Errorf("MonSupported() returned false, expected true") } expectedMonFeatures := map[MonResource][]string{MonResourceL3: []string{"llc_occupancy", "mbm_local_bytes", "mbm_total_bytes"}} if features := GetMonFeatures(); !cmp.Equal(features, expectedMonFeatures) { t.Fatalf("GetMonFeatures() returned %v, expected %v", features, expectedMonFeatures) } // Test creating monitoring groups cls, _ = GetClass("Guaranteed") mgName := "test_group" mgAnnotations := map[string]string{"a_key": "a_value"} mg, err := cls.CreateMonGroup(mgName, mgAnnotations) if err != nil { t.Fatalf("creating mon group failed: %v", err) } if n := mg.Name(); n != mgName { t.Errorf("MonGroup.Name() returned %q, expected %q", n, mgName) } if a := mg.GetAnnotations(); !cmp.Equal(a, mgAnnotations) { t.Errorf("MonGroup.GetAnnotations() returned %s, expected %s", a, mgAnnotations) } if n := mg.Parent().Name(); n != "Guaranteed" { t.Errorf("MonGroup.Parent().Name() returned %q, expected %q", n, "Guaranteed") } if _, ok := cls.GetMonGroup("non-existing-group"); ok { t.Errorf("unexpected success when querying non-existing group") } if _, ok := cls.GetMonGroup(mgName); !ok { t.Errorf("unexpected error when querying mon group: %v", err) } verifyGroupNames(cls.GetMonGroups(), []string{"predefined_group_live", mgName}) mgPath := rdt.classes["Guaranteed"].path("mon_groups", "goresctrl."+mgName) if _, err := os.Stat(mgPath); err != nil { t.Errorf("mon group directory not found: %v", err) } // Check that the monGroup.path() and relPath() methods work correctly mgi := rdt.classes["Guaranteed"].monGroups[mgName] if p := mgi.path("foo"); p != filepath.Join(mockFs.baseDir, "resctrl", "goresctrl.Guaranteed", "mon_groups", "goresctrl."+mgName, "foo") { t.Errorf("path() returned wrong path %q", p) } if p := mgi.relPath("foo"); p != filepath.Join("goresctrl.Guaranteed", "mon_groups", "goresctrl."+mgName, "foo") { t.Errorf("relPath() returned wrong path %q", p) } // Test deleting monitoring groups if err := cls.DeleteMonGroup(mgName); err != nil { t.Errorf("unexpected error when deleting mon group: %v", err) } if _, ok := cls.GetMonGroup("non-existing-group"); ok { t.Errorf("unexpected success when querying deleted group") } if _, err := os.Stat(mgPath); !os.IsNotExist(err) { t.Errorf("unexpected error when checking directory of deleted mon group: %v", err) } for _, n := range []string{"foo", "bar", "baz"} { if _, err := cls.CreateMonGroup(n, map[string]string{}); err != nil { t.Errorf("creating mon group failed: %v", err) } } if err := cls.DeleteMonGroups(); err != nil { t.Errorf("unexpected error when deleting all mon groups: %v", err) } if mgs := cls.GetMonGroups(); len(mgs) != 0 { t.Errorf("unexpected mon groups exist: %v", mgs) } // Verify assigning pids to monitor group mgName = "test_group_2" mockFs.initMockMonGroup("Guaranteed", mgName) cls, _ = GetClass("Guaranteed") mg, _ = cls.CreateMonGroup(mgName, nil) pids = []string{"10"} if err := mg.AddPids(pids...); err != nil { t.Errorf("MonGroup.AddPids() failed: %v", err) } if p, err := mg.GetPids(); err != nil { t.Errorf("MonGroup.GetPids() failed: %v", err) } else if !cmp.Equal(p, pids) { t.Errorf("MonGroup.GetPids() returned %s, expected %s", p, pids) } mockFs.verifyTextFile(rdt.classes["Guaranteed"].monGroups[mgName].relPath("tasks"), "10\n") // Verify monitoring functionality expected := MonData{ L3: MonL3Data{ 0: MonLeafData{ "llc_occupancy": 1, "mbm_local_bytes": 2, "mbm_total_bytes": 3, }, 1: MonLeafData{ "llc_occupancy": 11, "mbm_local_bytes": 12, "mbm_total_bytes": 13, }, 2: MonLeafData{ "llc_occupancy": 21, "mbm_local_bytes": 22, "mbm_total_bytes": 23, }, 3: MonLeafData{ "llc_occupancy": 31, "mbm_local_bytes": 32, "mbm_total_bytes": 33, }, }, } md := mg.GetMonData() if !cmp.Equal(md, expected) { t.Errorf("unexcpected monitoring data\nexpected:\n%s\nreceived:\n%s", utils.DumpJSON(expected), utils.DumpJSON(md)) } // // 3. Test discovery // if err := DiscoverClasses(""); err != nil { t.Fatalf("DiscoverClasses() failed unexpectedly") } classes = GetClasses() verifyGroupNames(classes, []string{"Guaranteed", "non_goresctrl.Group", RootClassName}) if err := DiscoverClasses("non_goresctrl."); err != nil { t.Fatalf("DiscoverClasses() failed unexpectedly") } classes = GetClasses() verifyGroupNames(classes, []string{"Group", RootClassName}) if err := DiscoverClasses("non-existing-prefix"); err != nil { t.Fatalf("DiscoverClasses() failed unexpectedly") } classes = GetClasses() verifyGroupNames(classes, []string{RootClassName}) } // TestConfig tests configuration parsing and resolving func TestConfig(t *testing.T) { type Schemata struct { l2 string l2code string l2data string l3 string l3code string l3data string mb string } type TC struct { name string fs string fsMountOpts string config string configErrRe string schemata map[string]Schemata } tcs := []TC{ // Testcase TC{ name: "Empty config", fs: "resctrl.full", config: "", schemata: map[string]Schemata{ "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", mb: "0=100;1=100;2=100;3=100", }, }, }, // Testcase TC{ name: "Complex config", fs: "resctrl.full", config: ` partitions: part-1: l3Allocation: all: 60% 1: "0xff000" 2: "9-15" mbAllocation: all: [100%] classes: class-1: l3Allocation: 100% class-2: l3Allocation: all: 100% 0-1: 10% 2: "0x70" mbAllocation: all: [40%] 3: [10%] part-2: l3Allocation: all: 39% 1: "0-10" 2: "0-6" mbAllocation: all: [50%] 1: [80%] 2: [100%] classes: class-3: l3Allocation: 100% mbAllocation: all: [40%] 0: [80%] class-4: l3Allocation: 50% mbAllocation: [100%] system/default: l3Allocation: 60% mbAllocation: [60%] part-3: l3Allocation: all: 1% 1: "0x800" 2: "7,8" mbAllocation: [20%] classes: class-5: l3Allocation: 100% mbAllocation: all: [100%] 0: [1%] `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=fff;1=ff000;2=fe00;3=fff", mb: "0=100;1=100;2=100;3=100", }, "class-2": Schemata{ l3: "0=3;1=1000;2=e000;3=fff", mb: "0=40;1=40;2=40;3=10", }, "class-3": Schemata{ l3: "0=7f000;1=7ff;2=7f;3=7f000", mb: "0=40;1=32;2=40;3=20", }, "class-4": Schemata{ l3: "0=f000;1=3f;2=f;3=f000", mb: "0=50;1=80;2=100;3=50", }, "system/default": Schemata{ l3: "0=1f000;1=7f;2=1f;3=1f000", mb: "0=30;1=48;2=60;3=30", }, "class-5": Schemata{ l3: "0=80000;1=800;2=180;3=80000", mb: "0=10;1=20;2=20;3=20", }, }, }, // Testcase TC{ name: "L3 CDP disabled", fs: "resctrl.nomb", config: ` partitions: part-1: l3Allocation: 0,1: unified: 60% code: 70% data: 50% 2,3: 40% classes: class-1: part-2: l3Allocation: 0,1: unified: 40% code: 30% data: 50% 2,3: 60% classes: class-2: system/default: l3Allocation: all: 100% 3: unified: 80% code: 60% data: 90% `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=fff;1=fff;2=ff;3=ff", }, "class-2": Schemata{ l3: "0=ff000;1=ff000;2=fff00;3=fff00", }, "system/default": Schemata{ l3: "0=ff000;1=ff000;2=fff00;3=3ff00", }, }, }, // Testcase TC{ name: "L3 CDP enabled", fs: "resctrl.nomb.cdp", config: ` partitions: part-1: l3Allocation: 0,1: unified: 60% code: 70% data: 50% 2,3: 40% classes: class-1: part-2: l3Allocation: 0,1: unified: 40% code: 30% data: 50% 2,3: 60% classes: class-2: "": l3Allocation: all: 100% 3: unified: 80% code: 60% data: 90% `, schemata: map[string]Schemata{ "class-1": Schemata{ l3code: "0=3fff;1=3fff;2=ff;3=ff", l3data: "0=3ff;1=3ff;2=ff;3=ff", }, "class-2": Schemata{ l3code: "0=fc000;1=fc000;2=fff00;3=fff00", l3data: "0=ffc00;1=ffc00;2=fff00;3=fff00", }, "system/default": Schemata{ l3code: "0=fc000;1=fc000;2=fff00;3=ff00", l3data: "0=ffc00;1=ffc00;2=fff00;3=7ff00", }, }, }, // Testcase TC{ name: "L3 optional", fs: "resctrl.nol3", config: ` options: l3: optional: true partitions: part-1: l3Allocation: 100% mbAllocation: [100%] classes: class-1: l3Allocation: 20% mbAllocation: [50%] `, schemata: map[string]Schemata{ "class-1": Schemata{ mb: "0=50;1=50;2=50;3=50", }, "system/default": Schemata{ mb: "0=100;1=100;2=100;3=100", }, }, }, // Testcase TC{ name: "Default L3 CAT", fs: "resctrl.full", config: ` options: partitions: part-1: mbAllocation: [100%] classes: class-1: mbAllocation: [50%] `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", mb: "0=50;1=50;2=50;3=50", }, "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", mb: "0=100;1=100;2=100;3=100", }, }, }, // Testcase TC{ name: "Default MBA", fs: "resctrl.full", config: ` options: partitions: part-1: l3Allocation: 100% classes: class-1: l3Allocation: 50% `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=3ff;1=3ff;2=3ff;3=3ff", mb: "0=100;1=100;2=100;3=100", }, "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", mb: "0=100;1=100;2=100;3=100", }, }, }, // Testcase TC{ name: "duplicate class names (fail)", fs: "resctrl.nomb", configErrRe: `"class-1" defined multiple times`, config: ` partitions: part-1: classes: class-1: part-2: classes: class-1: `, }, // Testcase TC{ name: "duplicate root class (fail)", fs: "resctrl.nomb", configErrRe: `"system/default" defined multiple times`, config: ` partitions: part-1: classes: "": part-2: classes: system/default: `, }, // Testcase TC{ name: "invalid class name", fs: "resctrl.nomb", configErrRe: `unqualified class name`, config: ` partitions: part-1: classes: "..": `, }, // Testcase TC{ name: "Invalid cache ids (fail)", fs: "resctrl.nomb", configErrRe: `failed to parse L3 allocation request for partition "part-1": invalid integer "a"`, config: ` partitions: part-1: l3Allocation: a: 100% `, }, // Testcase TC{ name: "L3 invalid allocation schema #3, missing unified (fail)", fs: "resctrl.nomb", configErrRe: `failed to parse L3 allocation request for partition "part-1": 'unified' not specified in cache schema`, config: ` partitions: part-1: l3Allocation: all: data: 100% `, }, // Testcase TC{ name: "L3 invalid allocation schema #4, missing code (fail)", fs: "resctrl.nomb", configErrRe: `failed to parse L3 allocation request for partition "part-1": 'data' specified but missing 'code' from cache schema`, config: ` partitions: part-1: l3Allocation: all: unified: 100% data: 100% `, }, // Testcase TC{ name: "L3 invalid allocation schema #5, missing data (fail)", fs: "resctrl.nomb", configErrRe: `failed to parse L3 allocation request for partition "part-1": 'code' specified but missing 'data' from cache schema`, config: ` partitions: part-1: l3Allocation: all: unified: 100% code: 100% `, }, // Testcase TC{ name: "L3 required (fail)", fs: "resctrl.nol3", configErrRe: `L3 cache allocation for "class-1" specified in configuration but not supported by system`, config: ` partitions: part-1: l3Allocation: 100% classes: class-1: l3Allocation: 20% `, }, // Testcase TC{ name: "MB optional", fs: "resctrl.nomb", config: ` options: mb: optional: true partitions: part-1: l3Allocation: 100% mbAllocation: [100%] classes: class-1: l3Allocation: 0-7 mbAllocation: [50%] `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=ff;1=ff;2=ff;3=ff", }, "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", }, }, }, // Testcase TC{ name: "MB required (fail)", fs: "resctrl.nomb", configErrRe: `memory bandwidth allocation for "class-1" specified in configuration but not supported by system`, config: ` partitions: part-1: mbAllocation: [100%] classes: class-1: mbAllocation: [50%] `, }, // Testcase TC{ name: "L3 mix rel and abs allocation in partition (fail)", fs: "resctrl.full", configErrRe: "error resolving L3 allocation for cache id 0: mixing absolute and relative allocations between partitions not supported", config: ` partitions: part-1: l3Allocation: "0xff" part-2: l3Allocation: 50% `, }, // Testcase TC{ name: "L3 mix rel and abs allocation in partition #2 (fail)", fs: "resctrl.full", configErrRe: "error resolving L3 allocation for cache id 0: mixing relative and absolute allocations between partitions not supported", config: ` partitions: part-1: l3Allocation: 50% part-2: l3Allocation: "0xff" `, }, // Testcase TC{ name: "L3 mix rel and abs allocation in classes", fs: "resctrl.nomb", config: ` partitions: part-1: l3Allocation: 100% classes: class-1: l3Allocation: all: 100% 1: 50% class-2: l3Allocation: all: 50% 1: "0x7" 2: "1-2" `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=fffff;1=3ff;2=fffff;3=fffff", }, "class-2": Schemata{ l3: "0=3ff;1=7;2=6;3=3ff", }, "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", }, }, }, // Testcase TC{ name: "L3 partial allocation", fs: "resctrl.nomb", config: ` partitions: part-1: l3Allocation: all: "21%" 1: "42%" 2: "63%" 3: "89%" classes: class-1: part-2: l3Allocation: all: "29%" 1: "8%" 2: "19%" 3: "11%" classes: class-2: `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=f;1=ff;2=1fff;3=3ffff", }, "class-2": Schemata{ l3: "0=3f0;1=300;2=e000;3=c0000", }, "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", }, }, }, // Testcase TC{ name: "L3 partition non-contiguous bitmask (fail)", fs: "resctrl.nomb", configErrRe: `failed to parse L3 allocation request for partition "part-1": invalid cache bitmask "0x2f": more than one continuous block of ones`, config: ` partitions: part-1: l3Allocation: all: "100%" 1: "0x2f" `, }, // Testcase TC{ name: "L3 overlapping partitions (fail)", fs: "resctrl.nomb", configErrRe: `overlapping L3 partition allocation requests for cache id 2`, config: ` partitions: part-1: l3Allocation: "0xff" part-2: l3Allocation: all: "0xff00" 2: "0xff80" `, }, // Testcase TC{ name: "L3 nan percentage in partition (fail)", fs: "resctrl.nomb", configErrRe: `failed to parse L3 allocation request for partition "part-1": strconv.ParseUint: parsing "1f": invalid syntax`, config: ` partitions: part-1: l3Allocation: "1f%" `, }, // Testcase TC{ name: "L3 percentage range in partition (fail)", fs: "resctrl.nomb", configErrRe: `invalid configuration: percentage ranges in partition allocation not supported`, config: ` partitions: part-1: l3Allocation: "50-100%" `, }, // Testcase TC{ name: "L3 missing for one partition (fail)", fs: "resctrl.full", configErrRe: `invalid configuration: some partitions \(part-2\) missing L3 "unified" allocation request`, config: ` partitions: part-1: l3Allocation: "50%" mbAllocation: ["100%"] part-2: mbAllocation: ["100%"] `, }, // Testcase TC{ name: "L3 percentage over 100 in partition (fail)", fs: "resctrl.nomb", configErrRe: `failed to parse L3 allocation request for partition "part-1": invalid percentage value "101%"`, config: ` partitions: part-1: l3Allocation: "101%" `, }, // Testcase TC{ name: "L3 missing cdp (fail)", fs: "resctrl.nomb", configErrRe: `some partitions \(part-2\) missing L3 "code" allocation request for cache id [0-3]`, config: ` partitions: part-1: l3Allocation: all: unified: "50%" code: "40%" data: "60%" part-2: l3Allocation: "50%" `, }, // Testcase TC{ name: "L3 total percentage over 100 (fail)", fs: "resctrl.nomb", configErrRe: `accumulated L3 "data" partition allocation requests for cache id [0-3] exceeds 100%`, config: ` partitions: part-1: l3Allocation: all: unified: "50%" code: "40%" data: "60%" part-2: l3Allocation: all: unified: "50%" code: "40%" data: "60%" `, }, // Testcase TC{ name: "L3 class allocation does not fit partition (fail)", fs: "resctrl.nomb", configErrRe: `bitmask 0x1ff00 \(0x1ff << 8\) does not fit basemask 0xff00`, config: ` partitions: part-1: l3Allocation: "0xff00" classes: class-1: l3Allocation: "0x1ff" `, }, // Testcase TC{ name: "L3 min cbm bits is respected", fs: "resctrl.nomb", config: ` partitions: part-1: l3Allocation: "100%" classes: class-1: l3Allocation: all: "1%" 1-2: "99-100%" `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=3;1=c0000;2=c0000;3=3", }, "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", }, }, }, // Testcase TC{ name: "L3 too few bits (fail)", fs: "resctrl.nomb", configErrRe: `bitmask 0x1ff00 \(0x1ff << 8\) does not fit basemask 0xff00`, config: ` partitions: part-1: l3Allocation: "0xff00" classes: class-1: l3Allocation: "0x1ff" `, }, // Testcase TC{ name: "L3 invalid percentage range in class (fail)", fs: "resctrl.nomb", configErrRe: `invalid configuration: failed to resolve L3 allocation for class "class-1": invalid percentage range`, config: ` partitions: part-1: l3Allocation: "100%" classes: class-1: l3Allocation: "0-101%" `, }, // Testcase TC{ name: "L3 missing from partition (fail)", fs: "resctrl.nomb", configErrRe: `L3 allocation missing from partition "part-1"`, config: ` partitions: part-1: classes: class-1: l3Allocation: "100%" `, }, // Testcase TC{ name: "MB allocation under minimum", fs: "resctrl.nol3", config: ` partitions: part-1: mbAllocation: ["1%"] classes: class-1: mbAllocation: ["100%"] `, schemata: map[string]Schemata{ "class-1": Schemata{ mb: "0=10;1=10;2=10;3=10", }, "system/default": Schemata{ mb: "0=100;1=100;2=100;3=100", }, }, }, // Testcase TC{ name: "L2, partial allocation", fs: "resctrl.l2", config: ` partitions: part-1: l2Allocation: all: 30% 1: 75% classes: class-1: part-2: l2Allocation: 0: 30% 1: unified: 20% classes: class-2: part-3: l2Allocation: 0: 40% 1: 5% classes: system/default: `, schemata: map[string]Schemata{ "class-1": Schemata{ l2: "0=3;1=3f", }, "class-2": Schemata{ l2: "0=c;1=40", }, "system/default": Schemata{ l2: "0=f0;1=80", }, }, }, // Testcase TC{ name: "L2 CDP", fs: "resctrl.l2cdp", config: ` partitions: part-1: l2Allocation: all: 42% 2: unified: 30% code: 20% data: 50% 3: unified: 30% code: 40% data: 50% l3Allocation: 30% classes: class-1: part-2: l2Allocation: all: 43% 2: unified: 70% code: 40% data: 30% 3: unified: 30% code: 60% data: 50% l3Allocation: 50% classes: class-2: l2Allocation: all: 80% 2: unified: 80% code: 60% data: 90% system/default: l3Allocation: 60% `, schemata: map[string]Schemata{ "class-1": Schemata{ l2code: "0=ff;1=ff;2=f;3=ff", l2data: "0=ff;1=ff;2=3ff;3=3ff", l3: "0=7", }, "class-2": Schemata{ l2code: "0=ff00;1=ff00;2=1f0;3=3ff00", l2data: "0=ff00;1=ff00;2=fc00;3=3fc00", l3: "0=1f8", }, "system/default": Schemata{ l2code: "0=1ff00;1=1ff00;2=ff0;3=fff00", l2data: "0=1ff00;1=1ff00;2=fc00;3=ffc00", l3: "0=78", }, }, }, // Testcase TC{ name: "L2 optional", fs: "resctrl.nomb", config: ` options: l2: optional: true partitions: part-1: l2Allocation: 50% l3Allocation: 50% classes: class-1: l2Allocation: 20% `, schemata: map[string]Schemata{ "class-1": Schemata{ l3: "0=3ff;1=3ff;2=3ff;3=3ff", }, "system/default": Schemata{ l3: "0=fffff;1=fffff;2=fffff;3=fffff", }, }, }, // Testcase TC{ name: "MB nan percentage value in partition (fail)", fs: "resctrl.nol3", configErrRe: `failed to resolve MB allocation for partition "part-1": strconv.ParseUint: parsing "xyz"`, config: ` partitions: part-1: mbAllocation: ["xyz%"] `, }, // Testcase TC{ name: "MB invalid percentage value in class (fail)", fs: "resctrl.nol3", configErrRe: `failed to resolve MB allocation for class "class-1":.*invalid syntax`, config: ` partitions: part-1: mbAllocation: ["100%"] classes: class-1: mbAllocation: ["1a%"] `, }, // Testcase TC{ name: "MB missing percentage value (fail)", fs: "resctrl.nol3", configErrRe: `missing '%' value from mbSchema`, config: ` partitions: part-1: mbAllocation: ["100MBps"] `, }, // Testcase TC{ name: "MB missing from partition (fail)", fs: "resctrl.nol3", configErrRe: `MB allocation missing from partition "part-1"`, config: ` partitions: part-1: classes: class-1: mbAllocation: ["100%"] `, }, // Testcase TC{ name: "MB MBps", fs: "resctrl.nol3.mbps", fsMountOpts: "mba_MBps", config: ` partitions: part-1: mbAllocation: ["50%", "1000MBps"] classes: class-1: mbAllocation: ["100%", "1500MBps"] part-2: mbAllocation: all: ["1000MBps"] # Unsupported values should just be ignored 0,1: [50, "1GBps", "500MBps"] classes: class-2: mbAllocation: ["750MBps"] `, schemata: map[string]Schemata{ "class-1": Schemata{ mb: "0=1000;1=1000;2=1000;3=1000", }, "class-2": Schemata{ mb: "0=500;1=500;2=750;3=750", }, "system/default": Schemata{ mb: "0=4294967295;1=4294967295;2=4294967295;3=4294967295", }, }, }, // Testcase TC{ name: "MB nan MBps value (fail)", fs: "resctrl.nol3.mbps", fsMountOpts: "mba_MBps", configErrRe: `failed to resolve MB allocation for partition "part-1":.* invalid syntax`, config: ` partitions: part-1: mbAllocation: ["0xffMBps"] `, }, // Testcase TC{ name: "MB missing MBps value (fail)", fs: "resctrl.nol3.mbps", fsMountOpts: "mba_MBps", configErrRe: `missing 'MBps' value from mbSchema`, config: ` partitions: part-1: mbAllocation: ["100%"] `, }, } verifySchemata := func(tc *TC) { for n, s := range tc.schemata { expected := "" if s.l2 != "" { expected += "L2:" + s.l2 + "\n" } if s.l2code != "" { expected += "L2CODE:" + s.l2code + "\n" } if s.l2data != "" { expected += "L2DATA:" + s.l2data + "\n" } if s.l3 != "" { expected += "L3:" + s.l3 + "\n" } if s.l3code != "" { expected += "L3CODE:" + s.l3code + "\n" } if s.l3data != "" { expected += "L3DATA:" + s.l3data + "\n" } if s.mb != "" { expected += "MB:" + s.mb + "\n" } if c, ok := rdt.classes[n]; !ok { t.Fatalf("verifySchemata: class %q does not exists in %v", n, rdt.classes) } else { verifyTextFile(t, c.path("schemata"), expected) } } if len(tc.schemata) != len(rdt.classes) { var a, b []string for n := range tc.schemata { a = append(a, n) } for n := range rdt.classes { b = append(b, n) } t.Fatalf("unexpected set of classes: expected %v, got %v", a, b) } } // Set group remove function so that mock groups can be removed groupRemoveFunc = os.RemoveAll for _, tc := range tcs { t.Logf("Running test case %q", tc.name) mockFs, err := newMockResctrlFs(t, tc.fs, tc.fsMountOpts) if err != nil { t.Fatalf("failed to set up mock resctrl fs: %v", err) } defer mockFs.delete() conf := parseTestConfig(t, tc.config) confDataOld, err := yaml.Marshal(conf) if err != nil { t.Fatalf("marshalling config failed: %v", err) } if err := Initialize(mockGroupPrefix); err != nil { t.Fatalf("resctrl initialization failed: %v", err) } err = SetConfig(conf, false) if tc.configErrRe != "" { if err == nil { t.Fatalf("resctrl configuration succeeded unexpectedly") } else { m, e := regexp.MatchString(tc.configErrRe, err.Error()) if e != nil { t.Fatalf("error in regexp matching: %v", e) } if !m { t.Fatalf("unexpected error message:\n %q\n does NOT match regexp\n %q", err.Error(), tc.configErrRe) } } } else { if err != nil { t.Fatalf("resctrl configuration failed: %v", err) } verifySchemata(&tc) } if confDataNew, err := yaml.Marshal(conf); err != nil { t.Fatalf("marshalling config failed: %v", err) } else if !cmp.Equal(confDataNew, confDataOld) { t.Fatalf("SetConfig altered config data:\n%s\nVS.\n%s", confDataOld, confDataNew) } } } func TestBitMap(t *testing.T) { // Test ListStr() testSet := map[bitmask]string{ 0x0: "", 0x1: "0", 0x2: "1", 0xf: "0-3", 0x555: "0,2,4,6,8,10", 0xaaa: "1,3,5,7,9,11", 0x1d1a: "1,3-4,8,10-12", 0xffffffffffffffff: "0-63", } for i, s := range testSet { // Test conversion to string listStr := i.listStr() if listStr != s { t.Errorf("from %#x expected %q, got %q", i, s, listStr) } // Test conversion from string b, err := listStrToBitmask(s) if err != nil { t.Errorf("unexpected err when converting %q: %v", s, err) } if b != i { t.Errorf("from %q expected %#x, got %#x", s, i, b) } } // Negative tests for ListStrToBitmask negTestSet := []string{ ",", "-", "1,", ",12", "-4", "0-", "13-13", "14-13", "a-2", "b", "3-c", "64", "1,2,,3", "1,2,3-", } for _, s := range negTestSet { b, err := listStrToBitmask(s) if err == nil { t.Errorf("expected err but got %#x when converting %q", b, s) } } // Test MarshalJSON if s, err := bitmask(10).MarshalJSON(); err != nil { } else if string(s) != `"0xa"` { t.Errorf(`expected "0xa" but returned %s`, s) } } func TestListStrToArray(t *testing.T) { testSet := map[string][]int{ "": {}, "0": {0}, "1": {1}, "0-3": {0, 1, 2, 3}, "4,2,0,6,10,8": {0, 2, 4, 6, 8, 10}, "1,3,5,7,9,11": {1, 3, 5, 7, 9, 11}, "1,3-4,10-12,8": {1, 3, 4, 8, 10, 11, 12}, } for s, expected := range testSet { // Test conversion from string to list of integers a, err := listStrToArray(s) if err != nil { t.Errorf("unexpected error when converting %q: %v", s, err) } if !cmp.Equal(a, expected) { t.Errorf("from %q expected %v, got %v", s, expected, a) } } // Negative test cases negTestSet := []string{ ",", "-", "1,", "256", "256-257", "0-256", ",12", "-4", "0-", "13-13", "14-13", "a-2", "b", "3-c", "1,2,,3", "1,2,3-", } for _, s := range negTestSet { a, err := listStrToArray(s) if err == nil { t.Errorf("expected err but got %v when converting %q", a, s) } } } // TestCacheAllocation tests the types implementing cacheAllocation interface func TestCacheAllocation(t *testing.T) { // Need to setup resctrl and initialize because pct allocations need // the "info" structure mockFs, err := newMockResctrlFs(t, "resctrl.nomb", "") if err != nil { t.Fatalf("failed to set up mock resctrl fs: %v", err) } defer mockFs.delete() if err := Initialize(mockGroupPrefix); err != nil { t.Fatalf("resctrl initialization failed: %v", err) } // Test absolute allocation abs := catAbsoluteAllocation(0x7) if res, err := abs.Overlay(0xf00, 1); err != nil { t.Errorf("unexpected error when overlaying catAbsoluteAllocation: %v", err) } else if res != 0x700 { t.Errorf("expected 0x700 but got %#x when overlaying catAbsoluteAllocation", res) } if _, err := abs.Overlay(0, 1); err == nil { t.Errorf("unexpected success when overlaying catAbsoluteAllocation with empty basemask") } if _, err := abs.Overlay(0x30, 1); err == nil { t.Errorf("unexpected success when overlaying too wide catAbsoluteAllocation") } if _, err := abs.Overlay(0xf0f, 1); err == nil { t.Errorf("unexpected success when overlaying catAbsoluteAllocation with non-contiguous basemask") } if _, err := catAbsoluteAllocation(0x1).Overlay(0x10, 2); err == nil { t.Errorf("unexpected success when overlaying catAbsoluteAllocation with too small basemask") } // Test percentage allocation if res, err := (catPctRangeAllocation{lowPct: 0, highPct: 100}).Overlay(0xff00, 4); err != nil { t.Errorf("unexpected error when overlaying catPctAllocation: %v", err) } else if res != 0xff00 { t.Errorf("expected 0xff00 but got %#x when overlaying catPctAllocation", res) } if res, err := (catPctRangeAllocation{lowPct: 99, highPct: 100}).Overlay(0xff00, 4); err != nil { t.Errorf("unexpected error when overlaying catPctAllocation: %v", err) } else if res != 0xf000 { t.Errorf("expected 0xf000 but got %#x when overlaying catPctAllocation", res) } if res, err := (catPctRangeAllocation{lowPct: 0, highPct: 1}).Overlay(0xff00, 4); err != nil { t.Errorf("unexpected error when overlaying catPctAllocation: %v", err) } else if res != 0xf00 { t.Errorf("expected 0xf00 but got %#x when overlaying catPctAllocation", res) } if res, err := (catPctRangeAllocation{lowPct: 20, highPct: 30}).Overlay(0x3ff00, 4); err != nil { t.Errorf("unexpected error when overlaying catPctAllocation: %v", err) } else if res != 0xf00 { t.Errorf("expected 0xf00 but got %#x when overlaying catPctAllocation", res) } if res, err := (catPctRangeAllocation{lowPct: 30, highPct: 60}).Overlay(0xf00, 4); err != nil { t.Errorf("unexpected error when overlaying catPctAllocation: %v", err) } else if res != 0xf00 { t.Errorf("expected 0xf00 but got %#x when overlaying catPctAllocation", res) } if _, err := (catPctRangeAllocation{lowPct: 20, highPct: 10}).Overlay(0xff00, 4); err == nil { t.Errorf("unexpected success when overlaying catPctAllocation of invalid percentage range") } if _, err := (catPctRangeAllocation{lowPct: 0, highPct: 100}).Overlay(0, 4); err == nil { t.Errorf("unexpected success when overlaying catPctAllocation of invalid percentage range") } } func TestCacheProportion(t *testing.T) { // Test percentage if a, err := CacheProportion("10%").parse(2); err != nil { t.Errorf("unexpected error when parsing cache allocation: %v", err) } else if a != catPctAllocation(10) { t.Errorf("expected 10%% but got %d%%", a) } if _, err := CacheProportion("1a%").parse(2); err == nil { t.Errorf("unexpected success when parsing percentage cache allocation") } if _, err := CacheProportion("101%").parse(2); err == nil { t.Errorf("unexpected success when parsing percentage cache allocation") } // Test percentage ranges if a, err := CacheProportion("10-20%").parse(2); err != nil { t.Errorf("unexpected error when parsing cache allocation: %v", err) } else if a != (catPctRangeAllocation{lowPct: 10, highPct: 20}) { t.Errorf("expected {10 20} but got %v", a) } if _, err := CacheProportion("a-100%").parse(2); err == nil { t.Errorf("unexpected success when parsing percentage range cache allocation") } if _, err := CacheProportion("0-1f%").parse(2); err == nil { t.Errorf("unexpected success when parsing percentage range cache allocation") } if _, err := CacheProportion("20-10%").parse(2); err == nil { t.Errorf("unexpected success when parsing percentage range cache allocation") } if _, err := CacheProportion("20-101%").parse(2); err == nil { t.Errorf("unexpected success when parsing percentage range cache allocation") } // Test bitmask if a, err := CacheProportion("0xf0").parse(2); err != nil { t.Errorf("unexpected error when parsing cache allocation: %v", err) } else if a != catAbsoluteAllocation(0xf0) { t.Errorf("expected 0xf0 but got %#x", a) } if _, err := CacheProportion("0x40").parse(2); err == nil { t.Errorf("unexpected success when parsing bitmask cache allocation") } if _, err := CacheProportion("0x11").parse(2); err == nil { t.Errorf("unexpected success when parsing bitmask cache allocation") } if _, err := CacheProportion("0xg").parse(2); err == nil { t.Errorf("unexpected success when parsing bitmask cache allocation") } // Test bit numbers if a, err := CacheProportion("3,4,5-7,8").parse(2); err != nil { t.Errorf("unexpected error when parsing cache allocation: %v", err) } else if a != catAbsoluteAllocation(0x1f8) { t.Errorf("expected 0x1f8 but got %#x", a) } if _, err := CacheProportion("3,5").parse(2); err == nil { t.Errorf("unexpected success when parsing bitmask cache allocation") } if _, err := CacheProportion("1").parse(2); err == nil { t.Errorf("unexpected success when parsing bitmask cache allocation") } if _, err := CacheProportion("3-x").parse(2); err == nil { t.Errorf("unexpected success when parsing bitmask cache allocation") } } func TestIsQualifiedClassName(t *testing.T) { tcs := map[string]bool{ "foo": true, RootClassName: true, RootClassAlias: true, ".": false, "..": false, "foo/bar": false, "foo\n": false, } for name, expected := range tcs { if r := IsQualifiedClassName(name); r != expected { t.Errorf("IsQualifiedClassName(%q) returned %v (expected %v)", name, r, expected) } } } golang-github-intel-goresctrl-0.3.0/pkg/sst/000077500000000000000000000000001432612301500207435ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/sst/_sst_types_amd64.go000066400000000000000000000022061432612301500244610ustar00rootroot00000000000000//go:build amd64 // +build amd64 /* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // This file is used for auto-generation of sst_types_amd64.go package sst // #include // #include // import "C" const ( ISST_IF_GET_PHY_ID = C.ISST_IF_GET_PHY_ID ISST_IF_IO_CMD = C.ISST_IF_IO_CMD ISST_IF_MBOX_COMMAND = C.ISST_IF_MBOX_COMMAND ) type isstIfCPUMaps C.struct_isst_if_cpu_maps type isstIfCPUMap C.struct_isst_if_cpu_map type isstIfIoReg C.struct_isst_if_io_reg type isstIfIoRegs C.struct_isst_if_io_regs type isstIfMboxCmd C.struct_isst_if_mbox_cmd type isstIfMboxCmds C.struct_isst_if_mbox_cmds golang-github-intel-goresctrl-0.3.0/pkg/sst/_sst_types_priv.go000066400000000000000000000054151432612301500245330ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // This file is used for auto-generation of sst_types_priv.go package sst // #include "tools/power/x86/intel-speed-select/isst.h" // #include "tools/arch/x86/include/asm/msr-index.h" // import "C" const ( // TDP (perf profile) related commands CONFIG_TDP = C.CONFIG_TDP CONFIG_TDP_GET_LEVELS_INFO = C.CONFIG_TDP_GET_LEVELS_INFO CONFIG_TDP_GET_TDP_CONTROL = C.CONFIG_TDP_GET_TDP_CONTROL CONFIG_TDP_SET_TDP_CONTROL = C.CONFIG_TDP_SET_TDP_CONTROL CONFIG_TDP_GET_TDP_INFO = C.CONFIG_TDP_GET_TDP_INFO CONFIG_TDP_GET_PWR_INFO = C.CONFIG_TDP_GET_PWR_INFO CONFIG_TDP_GET_TJMAX_INFO = C.CONFIG_TDP_GET_TJMAX_INFO CONFIG_TDP_GET_CORE_MASK = C.CONFIG_TDP_GET_CORE_MASK CONFIG_TDP_GET_TURBO_LIMIT_RATIOS = C.CONFIG_TDP_GET_TURBO_LIMIT_RATIOS CONFIG_TDP_SET_LEVEL = C.CONFIG_TDP_SET_LEVEL CONFIG_TDP_GET_UNCORE_P0_P1_INFO = C.CONFIG_TDP_GET_UNCORE_P0_P1_INFO CONFIG_TDP_GET_P1_INFO = C.CONFIG_TDP_GET_P1_INFO CONFIG_TDP_GET_MEM_FREQ = C.CONFIG_TDP_GET_MEM_FREQ CONFIG_TDP_GET_FACT_HP_TURBO_LIMIT_NUMCORES = C.CONFIG_TDP_GET_FACT_HP_TURBO_LIMIT_NUMCORES CONFIG_TDP_GET_FACT_HP_TURBO_LIMIT_RATIOS = C.CONFIG_TDP_GET_FACT_HP_TURBO_LIMIT_RATIOS CONFIG_TDP_GET_FACT_LP_CLIPPING_RATIO = C.CONFIG_TDP_GET_FACT_LP_CLIPPING_RATIO CONFIG_TDP_PBF_GET_CORE_MASK_INFO = C.CONFIG_TDP_PBF_GET_CORE_MASK_INFO CONFIG_TDP_PBF_GET_P1HI_P1LO_INFO = C.CONFIG_TDP_PBF_GET_P1HI_P1LO_INFO CONFIG_TDP_PBF_GET_TJ_MAX_INFO = C.CONFIG_TDP_PBF_GET_TJ_MAX_INFO CONFIG_TDP_PBF_GET_TDP_INFO = C.CONFIG_TDP_PBF_GET_TDP_INFO // CLOS related commands CONFIG_CLOS = C.CONFIG_CLOS CLOS_PM_QOS_CONFIG = C.CLOS_PM_QOS_CONFIG CLOS_PQR_ASSOC = C.CLOS_PQR_ASSOC CLOS_PM_CLOS = C.CLOS_PM_CLOS CLOS_STATUS = C.CLOS_STATUS MBOX_CMD_WRITE_BIT = C.MBOX_CMD_WRITE_BIT // PM commands READ_PM_CONFIG = C.READ_PM_CONFIG WRITE_PM_CONFIG = C.WRITE_PM_CONFIG PM_FEATURE = C.PM_FEATURE PM_QOS_INFO_OFFSET = C.PM_QOS_INFO_OFFSET PM_QOS_CONFIG_OFFSET = C.PM_QOS_CONFIG_OFFSET PM_CLOS_OFFSET = C.PM_CLOS_OFFSET PQR_ASSOC_OFFSET = C.PQR_ASSOC_OFFSET // Hardware P state interface MSR_PM_ENABLE = C.MSR_PM_ENABLE ) golang-github-intel-goresctrl-0.3.0/pkg/sst/gen_sst_types.sh000077500000000000000000000013011432612301500241630ustar00rootroot00000000000000#!/bin/bash -e set -o pipefail tmpfile="_sst_types_out.go" trap "rm -f $tmpfile" EXIT generate() { local target="$1" shift local copts=$@ echo "Generating $target..." go tool cgo -godefs -- $copts _"$target" | gofmt > "$tmpfile" mv "$tmpfile" "$target" } KERNEL_SRC_DIR="${KERNEL_SRC_DIR:-/usr/src/linux}" echo "INFO: using kernel sources at $KERNEL_SRC_DIR" # Generate types from Linux kernel (public) headers generate sst_types_amd64.go -I"$KERNEL_SRC_DIR/include/uapi" "-I$KERNEL_SRC_DIR/include" # Generate types from Linux kernel private headers generate sst_types_priv.go -I"$KERNEL_SRC_DIR" "-I$KERNEL_SRC_DIR/include -I$KERNEL_SRC_DIR/arch/x86/include/generated/" golang-github-intel-goresctrl-0.3.0/pkg/sst/sst.go000066400000000000000000000417071432612301500221140ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package sst import ( "fmt" stdlog "log" "os" grclog "github.com/intel/goresctrl/pkg/log" "github.com/intel/goresctrl/pkg/utils" ) // SstPackageInfo contains status of Intel Speed Select Technologies (SST) // for one CPU package type SstPackageInfo struct { // Package related to this SST info pkg *cpuPackageInfo // Gereric PP info PPSupported bool PPLocked bool PPVersion int PPCurrentLevel int PPMaxLevel int // Information about the currently active PP level CPSupported bool CPEnabled bool CPPriority CPPriorityType BFSupported bool BFEnabled bool BFCores utils.IDSet TFSupported bool TFEnabled bool ClosInfo [NumClos]SstClosInfo ClosCPUInfo ClosCPUSet } // NumClos is the number of CLOSes suported by SST-CP const NumClos = 4 // SstClosInfo contains parameters of one CLOS of SST-CP type SstClosInfo struct { EPP int ProportionalPriority int MinFreq int MaxFreq int DesiredFreq int } // CPPriorityType denotes the type CLOS priority ordering used in SST-CP type CPPriorityType int const ( Proportional CPPriorityType = 0 Ordered CPPriorityType = 1 ) // ClosCPUSet contains mapping from Clos id to a set of CPU ids type ClosCPUSet map[int]utils.IDSet const isstDevPath = "/dev/isst_interface" var sstlog grclog.Logger = grclog.NewLoggerWrapper(stdlog.New(os.Stderr, "[ sst ] ", 0)) // SstSupported returns true if Intel Speed Select Technologies (SST) is supported // by the system and can be interfaced via the Linux kernel device func SstSupported() bool { if _, err := os.Stat(isstDevPath); err != nil { if !os.IsNotExist(err) { sstlog.Warnf("failed to access sst device %q: %v", isstDevPath, err) } else { sstlog.Debugf("sst device %q does not exist", isstDevPath) } return false } return true } // Check that a list of CPUs belong to a given package func CheckPackageCpus(info *SstPackageInfo, cpus utils.IDSet) bool { return info.pkg.hasCpus(cpus) } // GetPackageInfo returns information of those packages given as a parameter // or all if none given. func GetPackageInfo(pkgs ...int) (map[int]*SstPackageInfo, error) { var numPkgs int var pkglist []int // Get topology information from sysfs packages, err := getOnlineCpuPackages() if err != nil { return nil, fmt.Errorf("failed to determine cpu topology: %w", err) } if len(pkgs) == 0 { for i := range packages { pkglist = append(pkglist, i) } } else { for _, i := range pkgs { if _, ok := packages[i]; !ok { return nil, fmt.Errorf("cpu package %d not present", i) } else { pkglist = append(pkglist, i) } } } numPkgs = len(pkglist) infomap := make(map[int]*SstPackageInfo, numPkgs) for _, i := range pkglist { info, err := getSinglePackageInfo(packages[i]) if err != nil { return nil, err } infomap[i] = &info } return infomap, nil } // getSinglePackageInfo returns information of the SST configuration of one cpu // package. func getSinglePackageInfo(pkg *cpuPackageInfo) (SstPackageInfo, error) { info := SstPackageInfo{} cpu := pkg.cpus[0] // We just need to pass one logical cpu from the pkg as an arg var rsp uint32 var err error // Read perf-profile feature info if rsp, err = sendMboxCmd(cpu, CONFIG_TDP, CONFIG_TDP_GET_LEVELS_INFO, 0, 0); err != nil { return info, fmt.Errorf("failed to read SST PP info: %v", err) } info.PPSupported = getBits(rsp, 31, 31) != 0 info.PPLocked = getBits(rsp, 24, 24) != 0 info.PPCurrentLevel = int(getBits(rsp, 16, 23)) info.PPMaxLevel = int(getBits(rsp, 8, 15)) info.PPVersion = int(getBits(rsp, 0, 7)) info.pkg = pkg // Forget about older hw with partial/convoluted support if info.PPVersion < 3 { sstlog.Infof("SST PP version %d (less than 3), giving up...") return info, nil } // Read the status of currently active perf-profile if !info.PPSupported { sstlog.Debugf("SST PP feature not supported, only profile level %d is valid", info.PPCurrentLevel) } if rsp, err = sendMboxCmd(cpu, CONFIG_TDP, CONFIG_TDP_GET_TDP_CONTROL, 0, uint32(info.PPCurrentLevel)); err != nil { return info, fmt.Errorf("failed to read SST BF/TF status: %v", err) } info.BFSupported = isBitSet(rsp, 1) info.BFEnabled = isBitSet(rsp, 17) info.TFSupported = isBitSet(rsp, 0) info.TFEnabled = isBitSet(rsp, 16) // Read base-frequency info if info.BFSupported { info.BFCores = utils.IDSet{} punitCoreIDs := make(map[utils.ID]utils.IDSet, len(pkg.cpus)) var maxPunitCore utils.ID for _, id := range pkg.cpus { pc, err := punitCPU(id) if err != nil { return info, err } punitCore := pc >> 1 if _, ok := punitCoreIDs[punitCore]; !ok { punitCoreIDs[punitCore] = utils.IDSet{} } punitCoreIDs[punitCore].Add(id) if punitCore > maxPunitCore { maxPunitCore = punitCore } } // Read out core masks in batches of 32 (32 bits per response) for i := 0; i <= int(maxPunitCore)/32; i++ { if rsp, err = sendMboxCmd(cpu, CONFIG_TDP, CONFIG_TDP_PBF_GET_CORE_MASK_INFO, 0, uint32(info.PPCurrentLevel+(i<<8))); err != nil { return info, fmt.Errorf("failed to read SST BF core mask (#%d): %v", i, err) } for bit := 0; bit < 32; bit++ { if isBitSet(rsp, uint32(bit)) { info.BFCores.Add(punitCoreIDs[utils.ID(i*32+bit)].Members()...) } } } } // Read core-power feature info if rsp, err = sendMboxCmd(cpu, READ_PM_CONFIG, PM_FEATURE, 0, 0); err != nil { return info, fmt.Errorf("failed to read SST CP info: %v", err) } info.CPSupported = isBitSet(rsp, 0) info.CPEnabled = isBitSet(rsp, 16) if info.CPSupported { if rsp, err = sendMboxCmd(cpu, CONFIG_CLOS, CLOS_PM_QOS_CONFIG, 0, 0); err != nil { return info, fmt.Errorf("failed to read SST CP status: %v", err) } info.CPPriority = CPPriorityType(getBits(rsp, 2, 2)) info.ClosCPUInfo = make(map[int]utils.IDSet, NumClos) for i := 0; i < NumClos; i++ { if rsp, err = sendClosCmd(cpu, CLOS_PM_CLOS, uint32(i), 0); err != nil { return info, fmt.Errorf("failed to read SST CLOS #%d info: %v", i, err) } info.ClosInfo[i] = SstClosInfo{ EPP: int(getBits(rsp, 0, 3)), ProportionalPriority: int(getBits(rsp, 4, 7)), MinFreq: int(getBits(rsp, 8, 15)), MaxFreq: int(getBits(rsp, 16, 23)), DesiredFreq: int(getBits(rsp, 24, 31)), } } for _, id := range pkg.cpus { closId, err := GetCPUClosID(id) if err != nil { continue } if info.ClosCPUInfo[closId] == nil { info.ClosCPUInfo[closId] = utils.NewIDSet(id) } else { info.ClosCPUInfo[closId].Add(id) } } } return info, nil } func getPunitCoreId(cpu utils.ID) (uint32, error) { p, err := punitCPU(cpu) if err != nil { return 0, err } punitCore := uint32(p) >> 1 return punitCore, nil } // GetCPUClosID returns the SST-CP CLOS id that a cpu is associated with. func GetCPUClosID(cpu utils.ID) (int, error) { punitCore, err := getPunitCoreId(cpu) if err != nil { return -1, fmt.Errorf("invalid core id %d for cpu %d: %v", punitCore, cpu, err) } rsp, err := sendClosCmd(cpu, CLOS_PQR_ASSOC, punitCore, 0) if err != nil { return -1, fmt.Errorf("failed to read CLOS number of cpu %d: %v", cpu, err) } return int(getBits(rsp, 16, 17)), nil } func getBits(val, i, j uint32) uint32 { lsb := i msb := j if i > j { lsb = j msb = i } return (val >> lsb) & ((1 << (msb - lsb + 1)) - 1) } func isBitSet(val, n uint32) bool { return val&(1< 0 { req = setBit(req, 2) } } if _, err := sendMboxCmd(cpu, CONFIG_CLOS, CLOS_PM_QOS_CONFIG, param, req); err != nil { return fmt.Errorf("failed to set SST-CP status: %v", err) } return nil } func enableCP(info *SstPackageInfo, cpu utils.ID) (uint32, error) { if err := writeClosPmQosConfig(info, cpu, true); err != nil { return 0, fmt.Errorf("Cannot set Clos status: %v", err) } return writePMConfig(info, cpu, true) } func disableCP(info *SstPackageInfo, cpu utils.ID) (uint32, error) { if err := writeClosPmQosConfig(info, cpu, false); err != nil { return 0, fmt.Errorf("Cannot set Clos status: %v", err) } return writePMConfig(info, cpu, false) } func setDefaultClosParam(info *SstPackageInfo, cpu utils.ID) error { defaultConfig := &SstClosInfo{MaxFreq: 255} for clos := 0; clos < 4; clos++ { if err := saveClos(defaultConfig, cpu, clos); err != nil { return err } } return nil } func assignCPU2Clos(info *SstPackageInfo, clos int) error { sstlog.Debugf("Setting Clos %d for cpus %v\n", clos, info.ClosCPUInfo[clos].Members()) for _, cpu := range info.ClosCPUInfo[clos].Members() { if err := associate2Clos(cpu, clos); err != nil { return fmt.Errorf("failed to associate cpu %d to clos %d: %v", cpu, clos, err) } } return nil } // ConfigureCP will allow caller to configure CPUs to various Clos. func ConfigureCP(info *SstPackageInfo, priority int, cpu2clos *ClosCPUSet) error { if priority < 0 || priority > 1 { return fmt.Errorf("Invalid CP priority value %d (valid 0 or 1)", priority) } if info.ClosCPUInfo == nil { info.ClosCPUInfo = make(map[int]utils.IDSet, len(*cpu2clos)) } for clos, cpus := range *cpu2clos { info.ClosCPUInfo[clos] = cpus.Clone() // Remove the CPU from other Clos if found for i := 0; i < NumClos; i++ { if i == clos { continue } for id := range cpus { info.ClosCPUInfo[i].Del(id) } } if err := assignCPU2Clos(info, clos); err != nil { return err } } info.CPPriority = CPPriorityType(priority) return nil } // ClosSetup stores the user supplied Clos information into punit func ClosSetup(info *SstPackageInfo, clos int, closInfo *SstClosInfo) error { if clos < 0 || clos >= NumClos { return fmt.Errorf("Invalid Clos value (%d)", clos) } if closInfo.MinFreq < 0 || closInfo.MinFreq > 255 { return fmt.Errorf("Invalid min freq (%d)", closInfo.MinFreq) } if closInfo.MaxFreq < 0 || closInfo.MaxFreq > 255 { return fmt.Errorf("Invalid max freq (%d)", closInfo.MaxFreq) } if closInfo.MinFreq > closInfo.MaxFreq { return fmt.Errorf("Min freq %d must be smaller than max freq %d", closInfo.MinFreq, closInfo.MaxFreq) } if closInfo.DesiredFreq < 0 || closInfo.DesiredFreq > 255 { return fmt.Errorf("Invalid value %d for desired freq", closInfo.DesiredFreq) } if closInfo.EPP < 0 || closInfo.EPP > 15 { return fmt.Errorf("Invalid value %d for EPP", closInfo.EPP) } if closInfo.ProportionalPriority < 0 || closInfo.ProportionalPriority > 15 { return fmt.Errorf("Invalid value %d for proportionalPriority", closInfo.ProportionalPriority) } info.ClosInfo[clos] = *closInfo return saveClos(&info.ClosInfo[clos], info.pkg.cpus[0], clos) } // ResetCPConfig will bring the system to a known state. This means that all // CLOS groups are reset to their default values, all package cores are assigned to // CLOS group 0 and ordered priority mode is enabled. func ResetCPConfig() error { infomap, err := GetPackageInfo() if err != nil { return err } for _, info := range infomap { for _, cpu := range info.pkg.cpus { if info.pkg.cpus[0] == cpu { if err := setDefaultClosParam(info, cpu); err != nil { return err } } if err := associate2Clos(cpu, 0); err != nil { return fmt.Errorf("failed to associate cpu %d to clos %d: %w", cpu, 0, err) } } } return nil } // EnableCP enables SST-CP feature func EnableCP(info *SstPackageInfo) error { if !info.CPSupported { return fmt.Errorf("SST CP not supported") } if len(info.ClosCPUInfo) == 0 { return fmt.Errorf("failed to enable CP: Clos to CPU mapping missing") } rsp, err := enableCP(info, info.pkg.cpus[0]) if err != nil { return fmt.Errorf("failed to enable SST-CP: %v", err) } info.CPSupported = isBitSet(rsp, 0) info.CPEnabled = isBitSet(rsp, 16) return nil } // DisableCP disables SST-CP feature func DisableCP(info *SstPackageInfo) error { if !info.CPSupported { return fmt.Errorf("SST CP not supported") } if info.TFEnabled { return fmt.Errorf("SST TF still enabled, disable it first.") } rsp, err := disableCP(info, info.pkg.cpus[0]) if err != nil { return fmt.Errorf("failed to disable SST-CP: %v", err) } info.CPSupported = isBitSet(rsp, 0) info.CPEnabled = isBitSet(rsp, 16) return nil } golang-github-intel-goresctrl-0.3.0/pkg/sst/sst_if.go000066400000000000000000000067421432612301500225720ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package sst //go:generate ./gen_sst_types.sh import ( "fmt" "os" "syscall" "unsafe" "github.com/intel/goresctrl/pkg/utils" ) // cpuMap holds the logical to punit cpu mapping table var cpuMap = make(map[utils.ID]utils.ID) // punitCPU returns the PUNIT CPU id corresponding a given Linux logical CPU func punitCPU(cpu utils.ID) (utils.ID, error) { if id, ok := cpuMap[cpu]; ok { return id, nil } id, err := getCPUMapping(cpu) if err == nil { cpuMap[cpu] = id } return id, err } // isstIoctl is a helper for executing ioctls on the linux isst_if device driver func isstIoctl(ioctl uintptr, req uintptr) error { f, err := os.Open(isstDevPath) if err != nil { return fmt.Errorf("failed to open isst device %q: %v", isstDevPath, err) } defer f.Close() if _, _, errno := syscall.Syscall(syscall.SYS_IOCTL, uintptr(f.Fd()), ioctl, req); errno != 0 { return errno } return nil } // getCPUMapping gets mapping of Linux logical CPU numbers to (package-specific) // PUNIT CPU number for one cpu. This is needed because the PUNIT CPU/core // numbering differs from the Linux kernel numbering (exposed via sysfs) which // is based on APIC. func getCPUMapping(cpu utils.ID) (utils.ID, error) { req := isstIfCPUMaps{ Cmd_count: 1, Cpu_map: [1]isstIfCPUMap{ {Logical_cpu: uint32(cpu)}, }, } if err := isstIoctl(ISST_IF_GET_PHY_ID, uintptr(unsafe.Pointer(&req))); err != nil { return -1, fmt.Errorf("failed to get CPU mapping for cpu %d: %v", cpu, err) } return utils.ID(req.Cpu_map[0].Physical_cpu), nil } // sendMboxCmd sends one mailbox command to PUNIT func sendMboxCmd(cpu utils.ID, cmd uint16, subCmd uint16, parameter uint32, reqData uint32) (uint32, error) { req := isstIfMboxCmds{ Cmd_count: 1, Mbox_cmd: [1]isstIfMboxCmd{ { Logical_cpu: uint32(cpu), Command: cmd, Sub_command: subCmd, Parameter: parameter, Req_data: reqData, }, }, } sstlog.Debugf("MBOX SEND cpu: %d cmd: %#02x sub: %#02x data: %#x", cpu, cmd, subCmd, reqData) if err := isstIoctl(ISST_IF_MBOX_COMMAND, uintptr(unsafe.Pointer(&req))); err != nil { return 0, fmt.Errorf("Mbox command failed with %v", err) } sstlog.Debugf("MBOX RECV data: %#x", req.Mbox_cmd[0].Resp_data) return req.Mbox_cmd[0].Resp_data, nil } // sendMMIOCmd sends one MMIO command to PUNIT func sendMMIOCmd(cpu utils.ID, reg uint32, value uint32, doWrite bool) (uint32, error) { var ReadWrite uint32 if doWrite { ReadWrite = 1 } req := isstIfIoRegs{ Req_count: 1, Io_reg: [1]isstIfIoReg{ { Logical_cpu: uint32(cpu), Reg: reg, Value: value, Read_write: ReadWrite, }, }, } sstlog.Debugf("MMIO SEND cpu: %d reg: %#x value: %#x write: %t", cpu, reg, value, doWrite) if err := isstIoctl(ISST_IF_IO_CMD, uintptr(unsafe.Pointer(&req))); err != nil { return 0, fmt.Errorf("MMIO command failed with %v", err) } sstlog.Debugf("MMIO RECV data: %#x", req.Io_reg[0].Value) return req.Io_reg[0].Value, nil } golang-github-intel-goresctrl-0.3.0/pkg/sst/sst_types_amd64.go000066400000000000000000000015221432612301500243220ustar00rootroot00000000000000// Code generated by cmd/cgo -godefs; DO NOT EDIT. // cgo -godefs -- -I/usr/src/linux/include/uapi -I/usr/src/linux/include _sst_types_amd64.go package sst const ( ISST_IF_GET_PHY_ID = 0xc008fe01 ISST_IF_IO_CMD = 0x4008fe02 ISST_IF_MBOX_COMMAND = 0xc008fe03 ) type isstIfCPUMaps struct { Cmd_count uint32 Cpu_map [1]isstIfCPUMap } type isstIfCPUMap struct { Logical_cpu uint32 Physical_cpu uint32 } type isstIfIoReg struct { Read_write uint32 Logical_cpu uint32 Reg uint32 Value uint32 } type isstIfIoRegs struct { Req_count uint32 Io_reg [1]isstIfIoReg } type isstIfMboxCmd struct { Logical_cpu uint32 Parameter uint32 Req_data uint32 Resp_data uint32 Command uint16 Sub_command uint16 Reserved uint32 } type isstIfMboxCmds struct { Cmd_count uint32 Mbox_cmd [1]isstIfMboxCmd } golang-github-intel-goresctrl-0.3.0/pkg/sst/sst_types_priv.go000066400000000000000000000026241432612301500243730ustar00rootroot00000000000000// Code generated by cmd/cgo -godefs; DO NOT EDIT. // cgo -godefs -- -I/usr/src/linux -I/usr/src/linux/include -I/usr/src/linux/arch/x86/include/generated/ _sst_types_priv.go package sst const ( CONFIG_TDP = 0x7f CONFIG_TDP_GET_LEVELS_INFO = 0x0 CONFIG_TDP_GET_TDP_CONTROL = 0x1 CONFIG_TDP_SET_TDP_CONTROL = 0x2 CONFIG_TDP_GET_TDP_INFO = 0x3 CONFIG_TDP_GET_PWR_INFO = 0x4 CONFIG_TDP_GET_TJMAX_INFO = 0x5 CONFIG_TDP_GET_CORE_MASK = 0x6 CONFIG_TDP_GET_TURBO_LIMIT_RATIOS = 0x7 CONFIG_TDP_SET_LEVEL = 0x8 CONFIG_TDP_GET_UNCORE_P0_P1_INFO = 0x9 CONFIG_TDP_GET_P1_INFO = 0xa CONFIG_TDP_GET_MEM_FREQ = 0xb CONFIG_TDP_GET_FACT_HP_TURBO_LIMIT_NUMCORES = 0x10 CONFIG_TDP_GET_FACT_HP_TURBO_LIMIT_RATIOS = 0x11 CONFIG_TDP_GET_FACT_LP_CLIPPING_RATIO = 0x12 CONFIG_TDP_PBF_GET_CORE_MASK_INFO = 0x20 CONFIG_TDP_PBF_GET_P1HI_P1LO_INFO = 0x21 CONFIG_TDP_PBF_GET_TJ_MAX_INFO = 0x22 CONFIG_TDP_PBF_GET_TDP_INFO = 0x23 CONFIG_CLOS = 0xd0 CLOS_PM_QOS_CONFIG = 0x2 CLOS_PQR_ASSOC = 0x0 CLOS_PM_CLOS = 0x1 CLOS_STATUS = 0x3 MBOX_CMD_WRITE_BIT = 0x8 READ_PM_CONFIG = 0x94 WRITE_PM_CONFIG = 0x95 PM_FEATURE = 0x3 PM_QOS_INFO_OFFSET = 0x0 PM_QOS_CONFIG_OFFSET = 0x4 PM_CLOS_OFFSET = 0x8 PQR_ASSOC_OFFSET = 0x20 MSR_PM_ENABLE = 0x770 ) golang-github-intel-goresctrl-0.3.0/pkg/sst/sysfs.go000066400000000000000000000044001432612301500224370ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package sst import ( "io/ioutil" "os" "path/filepath" "strconv" "strings" "github.com/intel/goresctrl/pkg/utils" ) type cpuPackageInfo struct { id int cpus []int } func (pkg *cpuPackageInfo) hasCpus(cpus utils.IDSet) bool { return utils.NewIDSetFromIntSlice(pkg.cpus...).Has(cpus.Members()...) } func getOnlineCpuPackages() (map[int]*cpuPackageInfo, error) { basePath := "/sys/bus/cpu/devices" files, err := ioutil.ReadDir(basePath) if err != nil { return nil, err } pkgs := make(map[int]*cpuPackageInfo) for _, file := range files { // Try to read siblings from topology raw, err := ioutil.ReadFile(filepath.Join(basePath, file.Name(), "topology/physical_package_id")) if os.IsNotExist(err) { // Offline -> topology information does not exist continue } else if err != nil { return nil, err } cpuId, err := strconv.Atoi(file.Name()[3:]) if err != nil { return nil, err } pkgId, err := strconv.Atoi(strings.TrimSpace(string(raw))) if err != nil { return nil, err } if _, ok := pkgs[pkgId]; !ok { pkgs[pkgId] = &cpuPackageInfo{id: pkgId} } pkgs[pkgId].cpus = append(pkgs[pkgId].cpus, cpuId) } return pkgs, nil } func isHWPEnabled() bool { status, err := utils.ReadMSR(0, MSR_PM_ENABLE) if err != nil { return false } return (status & 0xff) != 0 } func setCPUScalingMin2CPUInfoMinFreq(cpu utils.ID) error { freq, err := utils.GetCPUFreqValue(cpu, "cpuinfo_min_freq") if err != nil { return err } return utils.SetCPUScalingMinFreq(cpu, freq) } func setCPUScalingMin2CPUInfoMaxFreq(cpu utils.ID) error { freq, err := utils.GetCPUFreqValue(cpu, "cpuinfo_max_freq") if err != nil { return err } return utils.SetCPUScalingMinFreq(cpu, freq) } golang-github-intel-goresctrl-0.3.0/pkg/testutils/000077500000000000000000000000001432612301500221725ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/testutils/file.go000066400000000000000000000017011432612301500234370ustar00rootroot00000000000000// Copyright 2020-2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package testutils import ( "io/ioutil" "testing" ) func CreateTempFile(t *testing.T, data string) string { f, err := ioutil.TempFile("", "goresctrl-testutils-") if err != nil { t.Fatal(err) } if _, err := f.WriteString(data); err != nil { t.Fatal(err) } if err := f.Close(); err != nil { t.Fatal(err) } return f.Name() } golang-github-intel-goresctrl-0.3.0/pkg/testutils/verify.go000066400000000000000000000054771432612301500240420ustar00rootroot00000000000000// Copyright 2020-2021 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package testutils import ( "reflect" "strings" "testing" "github.com/hashicorp/go-multierror" ) // VerifyDeepEqual checks that two values (including structures) are equal, or else it fails the test. func VerifyDeepEqual(t *testing.T, valueName string, expectedValue interface{}, seenValue interface{}) bool { if reflect.DeepEqual(expectedValue, seenValue) { return true } t.Errorf("expected %s value %+v, got %+v", valueName, expectedValue, seenValue) return false } // VerifyError checks a (multi)error has expected properties, or else it fails the test. func VerifyError(t *testing.T, err error, expectedCount int, expectedSubstrings []string) bool { if expectedCount > 0 { if err == nil { t.Errorf("error expected, got nil") return false } merr, ok := err.(*multierror.Error) if !ok { if expectedCount > 1 { t.Errorf("expected %d errors, but got %#v instead of multierror", expectedCount, err) return false } // If exactly one error is expected, then err // is allowed to be any error, not just a // multierror. } else if len(merr.Errors) != expectedCount { t.Errorf("expected %d errors, but got %d: %v", expectedCount, len(merr.Errors), merr) return false } } else if expectedCount == 0 { if err != nil { t.Errorf("expected 0 errors, but got: %v", err) return false } } for _, substring := range expectedSubstrings { if !strings.Contains(err.Error(), substring) { t.Errorf("expected error with substring %#v, got \"%v\"", substring, err) } } return true } func VerifyNoError(t *testing.T, err error) bool { if err != nil { t.Errorf("expected no error, got %v", err) return false } return true } func VerifyStrings(t *testing.T, expected string, got string) bool { if expected != got { t.Errorf("Strings differ: expected %q, got %q", expected, got) return false } return true } func VerifyStringSlices(t *testing.T, expected []string, got []string) bool { if len(expected) != len(got) { t.Errorf("Expected string slice of length %d, got %d", len(expected), len(got)) return false } for i, es := range expected { if es != got[i] { t.Errorf("Slices differ: expected[%d]=%q, got[%d]=%q", i, es, i, got[i]) return false } } return true } golang-github-intel-goresctrl-0.3.0/pkg/utils/000077500000000000000000000000001432612301500212725ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/pkg/utils/idset.go000066400000000000000000000067211432612301500227370ustar00rootroot00000000000000/* Copyright 2019-2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package utils import ( "encoding/json" "fmt" "sort" "strconv" "strings" ) const ( // Unknown represents an unknown id. Unknown ID = -1 ) // ID is nn integer id, used to identify packages, CPUs, nodes, etc. type ID = int // IDSet is an unordered set of integer ids. type IDSet map[ID]struct{} // NewIDSet creates a new unordered set of (integer) ids. func NewIDSet(ids ...ID) IDSet { s := make(map[ID]struct{}) for _, id := range ids { s[id] = struct{}{} } return s } // NewIDSetFromIntSlice creates a new unordered set from an integer slice. func NewIDSetFromIntSlice(ids ...int) IDSet { s := make(map[ID]struct{}) for _, id := range ids { s[ID(id)] = struct{}{} } return s } // Clone returns a copy of this IdSet. func (s IDSet) Clone() IDSet { return NewIDSet(s.Members()...) } // Add adds the given ids into the set. func (s IDSet) Add(ids ...ID) { for _, id := range ids { s[id] = struct{}{} } } // Del deletes the given ids from the set. func (s IDSet) Del(ids ...ID) { if s != nil { for _, id := range ids { delete(s, id) } } } // Size returns the number of ids in the set. func (s IDSet) Size() int { return len(s) } // Has tests if all the ids are present in the set. func (s IDSet) Has(ids ...ID) bool { if s == nil { return false } for _, id := range ids { _, ok := s[id] if !ok { return false } } return true } // Members returns all ids in the set as a randomly ordered slice. func (s IDSet) Members() []ID { if s == nil { return []ID{} } ids := make([]ID, len(s)) idx := 0 for id := range s { ids[idx] = id idx++ } return ids } // SortedMembers returns all ids in the set as a sorted slice. func (s IDSet) SortedMembers() []ID { ids := s.Members() sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] }) return ids } // String returns the set as a string. func (s IDSet) String() string { return s.StringWithSeparator(",") } // StringWithSeparator returns the set as a string, separated with the given separator. func (s IDSet) StringWithSeparator(args ...string) string { if len(s) == 0 { return "" } var sep string if len(args) == 1 { sep = args[0] } else { sep = "," } str := "" t := "" for _, id := range s.SortedMembers() { str = str + t + strconv.Itoa(int(id)) t = sep } return str } // MarshalJSON is the JSON marshaller for IDSet. func (s IDSet) MarshalJSON() ([]byte, error) { return json.Marshal(s.String()) } // UnmarshalJSON is the JSON unmarshaller for IDSet. func (s *IDSet) UnmarshalJSON(data []byte) error { str := "" if err := json.Unmarshal(data, &str); err != nil { return fmt.Errorf("invalid IDSet entry '%s': %v", string(data), err) } *s = NewIDSet() if str == "" { return nil } for _, idstr := range strings.Split(str, ",") { id, err := strconv.ParseUint(idstr, 10, 0) if err != nil { return fmt.Errorf("invalid IDSet entry '%s': %v", idstr, err) } s.Add(ID(id)) } return nil } golang-github-intel-goresctrl-0.3.0/pkg/utils/json.go000066400000000000000000000015441432612301500225760ustar00rootroot00000000000000/* Copyright 2019 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package utils import ( "fmt" "sigs.k8s.io/yaml" ) // DumpJSON dumps a json-compatible struct in human-readable form func DumpJSON(r interface{}) string { out, err := yaml.Marshal(r) if err != nil { return fmt.Sprintf("!!!!!\nUnable to stringify %T: %v\n!!!!!", r, err) } return string(out) } golang-github-intel-goresctrl-0.3.0/pkg/utils/msr.go000066400000000000000000000021761432612301500224300ustar00rootroot00000000000000/* Copyright 2021 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package utils import ( "encoding/binary" "fmt" "os" ) func ReadMSR(cpu ID, msr int64) (uint64, error) { str := fmt.Sprintf("/dev/cpu/%d/msr", cpu) file, err := os.Open(str) if err != nil { return 0, err } defer file.Close() // Whence is the point of reference for offset // 0 = Beginning of file // 1 = Current position // 2 = End of file var whence int = 0 _, err = file.Seek(msr, whence) if err != nil { return 0, err } data := make([]byte, 8) _, err = file.Read(data) if err != nil { return 0, err } return binary.LittleEndian.Uint64(data), nil } golang-github-intel-goresctrl-0.3.0/pkg/utils/sort.go000066400000000000000000000022561432612301500226150ustar00rootroot00000000000000// Copyright 2020 Intel Corporation. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package utils import ( "sort" ) // SortUint64s sorts a slice of uint64 in increasing order. func SortUint64s(a []uint64) { sort.Sort(Uint64Slice(a)) } // Uint64Slice implmenents sort.Interface for a slice of uint64. type Uint64Slice []uint64 // Len returns the length of an UintSlice func (s Uint64Slice) Len() int { return len(s) } // Less returns true if element at 'i' is less than the element at 'j' func (s Uint64Slice) Less(i, j int) bool { return s[i] < s[j] } // Swap swaps the values of two elements func (s Uint64Slice) Swap(i, j int) { s[i], s[j] = s[j], s[i] } golang-github-intel-goresctrl-0.3.0/pkg/utils/sysfs.go000066400000000000000000000074751432612301500230050ustar00rootroot00000000000000/* Copyright 2022 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package utils import ( "fmt" "io/ioutil" "os" "strconv" "strings" ) const ( SysfsUncoreBasepath = "/sys/devices/system/cpu/intel_uncore_frequency/" ) func setCPUFreqValue(cpu ID, setting string, value int) error { path := fmt.Sprintf("/sys/devices/system/cpu/cpu%d/cpufreq/%s", cpu, setting) return writeFileInt(path, value) } // GetCPUFreqValue returns information of the currently used CPU frequency func GetCPUFreqValue(cpu ID, setting string) (int, error) { str := fmt.Sprintf("/sys/devices/system/cpu/cpu%d/cpufreq/%s", cpu, setting) raw, err := ioutil.ReadFile(str) if err != nil { return 0, err } value, err := strconv.Atoi(strings.TrimSpace(string(raw))) if err != nil { return 0, err } return value, nil } // SetCPUScalingMinFreq sets the scaling_min_freq value of a given CPU func SetCPUScalingMinFreq(cpu ID, freq int) error { return setCPUFreqValue(cpu, "scaling_min_freq", freq) } // SetCPUScalingMaxFreq sets the scaling_max_freq value of a given CPU func SetCPUScalingMaxFreq(cpu ID, freq int) error { return setCPUFreqValue(cpu, "scaling_max_freq", freq) } // SetCPUsScalingMinFreq sets the scaling_min_freq value of a given set of CPUs func SetCPUsScalingMinFreq(cpus []ID, freq int) error { for _, cpu := range cpus { if err := SetCPUScalingMinFreq(cpu, freq); err != nil { return err } } return nil } // SetCPUsScalingMaxFreq sets the scaling_max_freq value of a given set of CPUs func SetCPUsScalingMaxFreq(cpus []ID, freq int) error { for _, cpu := range cpus { if err := SetCPUScalingMaxFreq(cpu, freq); err != nil { return err } } return nil } // UncoreFreqAvailable returns true if the uncore frequency control functions are available. func UncoreFreqAvailable() bool { _, err := os.Stat(SysfsUncoreBasepath) return err == nil } // SetUncoreMinFreq sets the minimum uncore frequency of a CPU die. Frequency is specified in kHz. func SetUncoreMinFreq(pkg, die ID, freqKhz int) error { return setUncoreFreqValue(pkg, die, "min_freq_khz", freqKhz) } // SetUncoreMaxFreq sets the maximum uncore frequency of a CPU die. Frequency is specified in kHz. func SetUncoreMaxFreq(pkg, die ID, freqKhz int) error { return setUncoreFreqValue(pkg, die, "max_freq_khz", freqKhz) } func uncoreFreqPath(pkg, die ID, attribute string) string { return fmt.Sprintf(SysfsUncoreBasepath+"package_%02d_die_%02d/%s", pkg, die, attribute) } func getUncoreFreqValue(pkg, die ID, attribute string) (int, error) { return readFileInt(uncoreFreqPath(pkg, die, attribute)) } func setUncoreFreqValue(pkg, die ID, attribute string, value int) error { // Bounds checking if hwMinFreq, err := getUncoreFreqValue(pkg, die, "initial_min_freq_khz"); err != nil { return err } else if value < hwMinFreq { value = hwMinFreq } if hwMaxFreq, err := getUncoreFreqValue(pkg, die, "initial_max_freq_khz"); err != nil { return err } else if value > hwMaxFreq { value = hwMaxFreq } return writeFileInt(uncoreFreqPath(pkg, die, attribute), value) } func writeFileInt(path string, value int) error { return ioutil.WriteFile(path, []byte(strconv.Itoa(value)), 0644) } func readFileInt(path string) (int, error) { data, err := ioutil.ReadFile(path) if err != nil { return 0, err } i, err := strconv.ParseInt(strings.TrimSpace(string(data)), 10, 32) return int(i), err } golang-github-intel-goresctrl-0.3.0/test/000077500000000000000000000000001432612301500203305ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/000077500000000000000000000000001432612301500212415ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/rdt.go000066400000000000000000000015471432612301500223700ustar00rootroot00000000000000/* Copyright 2020 Intel Corporation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package rdtdata import ( "path/filepath" "runtime" ) var thisDir []string func init() { _, this, _, _ := runtime.Caller(0) thisDir = []string{filepath.Dir(this)} } // Path returns an absolute path to test data func Path(elem ...string) string { return filepath.Join(append(thisDir, elem...)...) } golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/000077500000000000000000000000001432612301500236605ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/000077500000000000000000000000001432612301500257375ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/cpus000066400000000000000000000000661432612301500266360ustar00rootroot0000000000000000000000,00000000,00000000,00000000,00000000,00000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/cpus_list000066400000000000000000000000011432612301500276560ustar00rootroot00000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mode000066400000000000000000000000121432612301500265770ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/000077500000000000000000000000001432612301500275215ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_00/000077500000000000000000000000001432612301500311475ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500336210ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500341360ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500341670ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_01/000077500000000000000000000000001432612301500311505ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500336220ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500341370ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500341700ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_02/000077500000000000000000000000001432612301500311515ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500336230ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500341400ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500341710ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_020 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_03/000077500000000000000000000000001432612301500311525ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500336240ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500341410ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500341720ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_data/mon_L3_030 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/000077500000000000000000000000001432612301500301275ustar00rootroot00000000000000non_goresctrl.group/000077500000000000000000000000001432612301500340615ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groupscpus000066400000000000000000000000661432612301500347600ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group00000000,00000000,00000000,00000000,00000000,00000000 cpus_list000066400000000000000000000000011432612301500360000ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group mon_data/000077500000000000000000000000001432612301500356435ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.groupmon_L3_00/000077500000000000000000000000001432612301500372715ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500420220ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500423370ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500423700ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mon_L3_01/000077500000000000000000000000001432612301500372725ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500420230ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500423400ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500423710ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mon_L3_02/000077500000000000000000000000001432612301500372735ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500420240ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500423410ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500423720ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mon_L3_03/000077500000000000000000000000001432612301500372745ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500420250ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500423420ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500423730ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 tasks000066400000000000000000000000001432612301500351170ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/mon_groups/non_goresctrl.groupgolang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/schemata000066400000000000000000000000761432612301500274520ustar00rootroot00000000000000L3:0=fffff;1=fffff;2=fffff;3=fffff MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/size000066400000000000000000000001121432612301500266260ustar00rootroot00000000000000L3:0=57671680;1=57671680;2=57671680;3=57671680 MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/Guaranteed/tasks000066400000000000000000000000001432612301500267750ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/cpus000066400000000000000000000000661432612301500245570ustar00rootroot00000000000000ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/cpus_list000066400000000000000000000000061432612301500256040ustar00rootroot000000000000000-191 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/000077500000000000000000000000001432612301500277425ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/cpus000066400000000000000000000000661432612301500306410ustar00rootroot0000000000000000000000,00000000,00000000,00000000,00000000,00000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/cpus_list000066400000000000000000000000011432612301500316610ustar00rootroot00000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mode000066400000000000000000000000121432612301500306020ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/000077500000000000000000000000001432612301500315245ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_00/000077500000000000000000000000001432612301500331525ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500356240ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500361410ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500361720ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_01/000077500000000000000000000000001432612301500331535ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500356250ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500361420ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500361730ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_02/000077500000000000000000000000001432612301500331545ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500356260ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500361430ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500361740ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_020 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_03/000077500000000000000000000000001432612301500331555ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500356270ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500361440ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500361750ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_data/mon_L3_030 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/000077500000000000000000000000001432612301500321325ustar00rootroot00000000000000goresctrl.predefined_group_empty/000077500000000000000000000000001432612301500406155ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groupscpus000066400000000000000000000000661432612301500415140ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty00000000,00000000,00000000,00000000,00000000,00000000 cpus_list000066400000000000000000000000011432612301500425340ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty mon_data/000077500000000000000000000000001432612301500423775ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_emptymon_L3_00/000077500000000000000000000000001432612301500440255ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_datallc_occupancy000066400000000000000000000000021432612301500465560ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500470730ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500471240ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_000 mon_L3_01/000077500000000000000000000000001432612301500440265ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_datallc_occupancy000066400000000000000000000000021432612301500465570ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500470740ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500471250ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_010 mon_L3_02/000077500000000000000000000000001432612301500440275ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_datallc_occupancy000066400000000000000000000000021432612301500465600ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500470750ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500471260ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_020 mon_L3_03/000077500000000000000000000000001432612301500440305ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_datallc_occupancy000066400000000000000000000000021432612301500465610ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500470760ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500471270ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_empty/mon_data/mon_L3_030 tasks000066400000000000000000000000001432612301500416530ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_emptygoresctrl.predefined_group_live/000077500000000000000000000000001432612301500404165ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groupscpus000066400000000000000000000000661432612301500413150ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live00000000,00000000,00000000,00000000,00000000,00000000 cpus_list000066400000000000000000000000011432612301500423350ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live mon_data/000077500000000000000000000000001432612301500422005ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_livemon_L3_00/000077500000000000000000000000001432612301500436265ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_datallc_occupancy000066400000000000000000000000021432612301500463570ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500466740ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500467250ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_000 mon_L3_01/000077500000000000000000000000001432612301500436275ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_datallc_occupancy000066400000000000000000000000021432612301500463600ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500466750ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500467260ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_010 mon_L3_02/000077500000000000000000000000001432612301500436305ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_datallc_occupancy000066400000000000000000000000021432612301500463610ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500466760ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500467270ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_020 mon_L3_03/000077500000000000000000000000001432612301500436315ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_datallc_occupancy000066400000000000000000000000021432612301500463620ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500466770ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500467300ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live/mon_data/mon_L3_030 tasks000066400000000000000000000000041432612301500414600ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/goresctrl.predefined_group_live100 non_goresctrl.group/000077500000000000000000000000001432612301500360645ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groupscpus000066400000000000000000000000661432612301500367630ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group00000000,00000000,00000000,00000000,00000000,00000000 cpus_list000066400000000000000000000000011432612301500400030ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group mon_data/000077500000000000000000000000001432612301500376465ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.groupmon_L3_00/000077500000000000000000000000001432612301500412745ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500440250ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500443420ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500443730ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mon_L3_01/000077500000000000000000000000001432612301500412755ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500440260ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500443430ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500443740ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mon_L3_02/000077500000000000000000000000001432612301500412765ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500440270ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500443440ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500443750ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mon_L3_03/000077500000000000000000000000001432612301500412775ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500440300ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500443450ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500443760ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 tasks000066400000000000000000000000001432612301500371220ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/mon_groups/non_goresctrl.groupgolang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/schemata000066400000000000000000000000761432612301500314550ustar00rootroot00000000000000L3:0=fffff;1=fffff;2=fffff;3=fffff MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/size000066400000000000000000000001121432612301500306310ustar00rootroot00000000000000L3:0=57671680;1=57671680;2=57671680;3=57671680 MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Guaranteed/tasks000066400000000000000000000000001432612301500310000ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/000077500000000000000000000000001432612301500267335ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/cpus000066400000000000000000000000661432612301500276320ustar00rootroot0000000000000000000000,00000000,00000000,00000000,00000000,00000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/cpus_list000066400000000000000000000000011432612301500306520ustar00rootroot00000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mode000066400000000000000000000000121432612301500275730ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/000077500000000000000000000000001432612301500305155ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_00/000077500000000000000000000000001432612301500321435ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500346150ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500351320ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500351630ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_01/000077500000000000000000000000001432612301500321445ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500346160ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500351330ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500351640ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_02/000077500000000000000000000000001432612301500321455ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500346170ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500351340ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500351650ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_020 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_03/000077500000000000000000000000001432612301500321465ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500346200ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500351350ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500351660ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_data/mon_L3_030 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/000077500000000000000000000000001432612301500311235ustar00rootroot00000000000000non_goresctrl.group/000077500000000000000000000000001432612301500350555ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groupscpus000066400000000000000000000000661432612301500357540ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group00000000,00000000,00000000,00000000,00000000,00000000 cpus_list000066400000000000000000000000011432612301500367740ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group mon_data/000077500000000000000000000000001432612301500366375ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.groupmon_L3_00/000077500000000000000000000000001432612301500402655ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500430160ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500433330ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500433640ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mon_L3_01/000077500000000000000000000000001432612301500402665ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500430170ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500433340ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500433650ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mon_L3_02/000077500000000000000000000000001432612301500402675ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500430200ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500433350ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500433660ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mon_L3_03/000077500000000000000000000000001432612301500402705ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500430210ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500433360ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500433670ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 tasks000066400000000000000000000000001432612301500361130ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/mon_groups/non_goresctrl.groupgolang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/schemata000066400000000000000000000000761432612301500304460ustar00rootroot00000000000000L3:0=fffff;1=fffff;2=fffff;3=fffff MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/size000066400000000000000000000001121432612301500276220ustar00rootroot00000000000000L3:0=57671680;1=57671680;2=57671680;3=57671680 MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/goresctrl.Stale/tasks000066400000000000000000000000001432612301500277710ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/000077500000000000000000000000001432612301500246135ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3/000077500000000000000000000000001432612301500250715ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3/bit_usage000066400000000000000000000001351432612301500267550ustar00rootroot000000000000000=XXSSSSSSSSSSSSSSSSSS;1=XXSSSSSSSSSSSSSSSSSS;2=XXSSSSSSSSSSSSSSSSSS;3=XXSSSSSSSSSSSSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3/cbm_mask000066400000000000000000000000061432612301500265640ustar00rootroot00000000000000fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3/min_cbm_bits000066400000000000000000000000021432612301500274310ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3/num_closids000066400000000000000000000000031432612301500273240ustar00rootroot0000000000000016 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3/shareable_bits000066400000000000000000000000061432612301500277570ustar00rootroot00000000000000c0000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3_MON/000077500000000000000000000000001432612301500256025ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3_MON/max_threshold_occupancy000066400000000000000000000000061432612301500324260ustar00rootroot0000000000000098304 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3_MON/mon_features000066400000000000000000000000561432612301500302150ustar00rootroot00000000000000llc_occupancy mbm_total_bytes mbm_local_bytes golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/L3_MON/num_rmids000066400000000000000000000000041432612301500275140ustar00rootroot00000000000000192 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/MB/000077500000000000000000000000001432612301500251115ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/MB/bandwidth_gran000066400000000000000000000000031432612301500300000ustar00rootroot0000000000000010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/MB/delay_linear000066400000000000000000000000021432612301500274540ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/MB/min_bandwidth000066400000000000000000000000031432612301500276340ustar00rootroot0000000000000010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/MB/num_closids000066400000000000000000000000021432612301500273430ustar00rootroot000000000000008 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/info/last_cmd_status000066400000000000000000000000031432612301500277200ustar00rootroot00000000000000ok golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mode000066400000000000000000000000121432612301500245200ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/000077500000000000000000000000001432612301500254425ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_00/000077500000000000000000000000001432612301500270705ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_00/llc_occupancy000066400000000000000000000000111432612301500316210ustar00rootroot0000000000000032440320 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_00/mbm_local_bytes000066400000000000000000000000111432612301500321360ustar00rootroot0000000000000048365568 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_00/mbm_total_bytes000066400000000000000000000000121432612301500321700ustar00rootroot00000000000000264830976 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_01/000077500000000000000000000000001432612301500270715ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_01/llc_occupancy000066400000000000000000000000111432612301500316220ustar00rootroot0000000000000028901376 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_01/mbm_local_bytes000066400000000000000000000000101432612301500321360ustar00rootroot000000000000003342336 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_01/mbm_total_bytes000066400000000000000000000000121432612301500321710ustar00rootroot00000000000000208404480 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_02/000077500000000000000000000000001432612301500270725ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_02/llc_occupancy000066400000000000000000000000111432612301500316230ustar00rootroot0000000000000034406400 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_02/mbm_local_bytes000066400000000000000000000000121432612301500321410ustar00rootroot00000000000000603881472 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_02/mbm_total_bytes000066400000000000000000000000121432612301500321720ustar00rootroot00000000000000974782464 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_03/000077500000000000000000000000001432612301500270735ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_03/llc_occupancy000066400000000000000000000000111432612301500316240ustar00rootroot0000000000000031260672 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_03/mbm_local_bytes000066400000000000000000000000121432612301500321420ustar00rootroot00000000000000693239808 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_data/mon_L3_03/mbm_total_bytes000066400000000000000000000000121432612301500321730ustar00rootroot00000000000000760479744 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/000077500000000000000000000000001432612301500260505ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/000077500000000000000000000000001432612301500275035ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/cpus000066400000000000000000000000661432612301500304020ustar00rootroot0000000000000000000000,00000000,00000000,00000000,00000000,00000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/cpus_list000066400000000000000000000000011432612301500314220ustar00rootroot00000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/000077500000000000000000000000001432612301500312655ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_00/000077500000000000000000000000001432612301500327135ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500353650ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_001 mbm_local_bytes000066400000000000000000000000021432612301500357020ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_002 mbm_total_bytes000066400000000000000000000000021432612301500357330ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_003 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_01/000077500000000000000000000000001432612301500327145ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000031432612301500353670ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0111 mbm_local_bytes000066400000000000000000000000031432612301500357040ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0112 mbm_total_bytes000066400000000000000000000000031432612301500357350ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0113 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_02/000077500000000000000000000000001432612301500327155ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000031432612301500353700ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0221 mbm_local_bytes000066400000000000000000000000031432612301500357050ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0222 mbm_total_bytes000066400000000000000000000000031432612301500357360ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0223 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_03/000077500000000000000000000000001432612301500327165ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000031432612301500353710ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0331 mbm_local_bytes000066400000000000000000000000031432612301500357060ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0332 mbm_total_bytes000066400000000000000000000000031432612301500357370ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/mon_data/mon_L3_0333 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/example/tasks000066400000000000000000000000021432612301500305430ustar00rootroot000000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/000077500000000000000000000000001432612301500320615ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/cpus000066400000000000000000000000661432612301500327600ustar00rootroot0000000000000000000000,00000000,00000000,00000000,00000000,00000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/cpus_list000066400000000000000000000000011432612301500340000ustar00rootroot00000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/000077500000000000000000000000001432612301500336435ustar00rootroot00000000000000mon_L3_00/000077500000000000000000000000001432612301500352125ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500377430ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500402600ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500403110ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_000 mon_L3_01/000077500000000000000000000000001432612301500352135ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500377440ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500402610ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500403120ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_010 mon_L3_02/000077500000000000000000000000001432612301500352145ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500377450ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500402620ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500403130ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_020 mon_L3_03/000077500000000000000000000000001432612301500352155ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_datallc_occupancy000066400000000000000000000000021432612301500377460ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500402630ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500403140ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/mon_data/mon_L3_030 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/mon_groups/non_goresctrl.group/tasks000066400000000000000000000000001432612301500331170ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/000077500000000000000000000000001432612301500276315ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/cpus000066400000000000000000000000661432612301500305300ustar00rootroot0000000000000000000000,00000000,00000000,00000000,00000000,00000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/cpus_list000066400000000000000000000000011432612301500315500ustar00rootroot00000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mode000066400000000000000000000000121432612301500304710ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/000077500000000000000000000000001432612301500314135ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_00/000077500000000000000000000000001432612301500330415ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500355130ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_000 mbm_local_bytes000066400000000000000000000000021432612301500360300ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_000 mbm_total_bytes000066400000000000000000000000021432612301500360610ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_01/000077500000000000000000000000001432612301500330425ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500355140ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_010 mbm_local_bytes000066400000000000000000000000021432612301500360310ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_010 mbm_total_bytes000066400000000000000000000000021432612301500360620ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_02/000077500000000000000000000000001432612301500330435ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500355150ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_020 mbm_local_bytes000066400000000000000000000000021432612301500360320ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_020 mbm_total_bytes000066400000000000000000000000021432612301500360630ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_020 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_03/000077500000000000000000000000001432612301500330445ustar00rootroot00000000000000llc_occupancy000066400000000000000000000000021432612301500355160ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_030 mbm_local_bytes000066400000000000000000000000021432612301500360330ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_030 mbm_total_bytes000066400000000000000000000000021432612301500360640ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/mon_data/mon_L3_030 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/schemata000066400000000000000000000000761432612301500313440ustar00rootroot00000000000000L3:0=fffff;1=fffff;2=fffff;3=fffff MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/size000066400000000000000000000001121432612301500305200ustar00rootroot00000000000000L3:0=57671680;1=57671680;2=57671680;3=57671680 MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/non_goresctrl.Group/tasks000066400000000000000000000000001432612301500306670ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/schemata000066400000000000000000000000761432612301500253730ustar00rootroot00000000000000L3:0=fffff;1=fffff;2=fffff;3=fffff MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/size000066400000000000000000000001121432612301500245470ustar00rootroot00000000000000L3:0=57671680;1=57671680;2=57671680;3=57671680 MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.full/tasks000066400000000000000000000004161432612301500247310ustar00rootroot000000000000001 2 3 4 6 8 10 11 12 13 14 15 16 17 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 49 50 51 52 53 54 55 56 57 58 59 60 61 62 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 99 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/000077500000000000000000000000001432612301500232335ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/cpus000066400000000000000000000000021432612301500241200ustar00rootroot00000000000000f golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/cpus_list000066400000000000000000000000041432612301500251550ustar00rootroot000000000000000-3 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/000077500000000000000000000000001432612301500241665ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/L2/000077500000000000000000000000001432612301500244435ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/L2/bit_usage000066400000000000000000000000261432612301500263260ustar00rootroot000000000000000=SSSSSSSS;1=SSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/L2/cbm_mask000066400000000000000000000000031432612301500261330ustar00rootroot00000000000000ff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/L2/min_cbm_bits000066400000000000000000000000021432612301500270030ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/L2/num_closids000066400000000000000000000000021432612301500266750ustar00rootroot000000000000004 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/L2/shareable_bits000066400000000000000000000000021432612301500273250ustar00rootroot000000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/info/last_cmd_status000066400000000000000000000000031432612301500272730ustar00rootroot00000000000000ok golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/mode000066400000000000000000000000121432612301500240730ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/schemata000066400000000000000000000000151432612301500247370ustar00rootroot00000000000000L2:0=ff;1=ff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/size000066400000000000000000000000271432612301500241270ustar00rootroot00000000000000L2:0=1048576;1=1048576 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2/tasks000066400000000000000000000004161432612301500243040ustar00rootroot000000000000001 2 3 4 6 8 10 11 12 13 14 15 16 17 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 49 50 51 52 53 54 55 56 57 58 59 60 61 62 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 99 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/000077500000000000000000000000001432612301500237225ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/cpus000066400000000000000000000000031432612301500246100ustar00rootroot00000000000000ff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/cpus_list000066400000000000000000000000041432612301500256440ustar00rootroot000000000000000-7 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/000077500000000000000000000000001432612301500246555ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2CODE/000077500000000000000000000000001432612301500255655ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2CODE/bit_usage000066400000000000000000000001341432612301500274500ustar00rootroot000000000000000=SSSSSSSSSSSSSSSSSSSS;1=SSSSSSSSSSSSSSSSSSSS;2=SSSSSSSSSSSSSSSSSSSS;3=SSSSSSSSSSSSSSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2CODE/cbm_mask000066400000000000000000000000061432612301500272600ustar00rootroot00000000000000fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2CODE/min_cbm_bits000066400000000000000000000000021432612301500301250ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2CODE/num_closids000066400000000000000000000000021432612301500300170ustar00rootroot000000000000004 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2CODE/shareable_bits000066400000000000000000000000021432612301500304470ustar00rootroot000000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2DATA/000077500000000000000000000000001432612301500255645ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2DATA/bit_usage000066400000000000000000000001341432612301500274470ustar00rootroot000000000000000=SSSSSSSSSSSSSSSSSSSS;1=SSSSSSSSSSSSSSSSSSSS;2=SSSSSSSSSSSSSSSSSSSS;3=SSSSSSSSSSSSSSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2DATA/cbm_mask000066400000000000000000000000061432612301500272570ustar00rootroot00000000000000fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2DATA/min_cbm_bits000066400000000000000000000000021432612301500301240ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2DATA/num_closids000066400000000000000000000000021432612301500300160ustar00rootroot000000000000004 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L2DATA/shareable_bits000066400000000000000000000000021432612301500304460ustar00rootroot000000000000000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L3/000077500000000000000000000000001432612301500251335ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L3/bit_usage000066400000000000000000000000171432612301500270160ustar00rootroot000000000000000=SXSSSSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L3/cbm_mask000066400000000000000000000000041432612301500266240ustar00rootroot00000000000000fff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L3/min_cbm_bits000066400000000000000000000000021432612301500274730ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L3/num_closids000066400000000000000000000000021432612301500273650ustar00rootroot000000000000004 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/L3/shareable_bits000066400000000000000000000000041432612301500300170ustar00rootroot00000000000000400 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/info/last_cmd_status000066400000000000000000000000031432612301500277620ustar00rootroot00000000000000ok golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/mode000066400000000000000000000000121432612301500245620ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/schemata000066400000000000000000000001351432612301500254310ustar00rootroot00000000000000 L3:0=00fff L2DATA:0=fffff;1=fffff;2=fffff;3=fffff L2CODE:0=fffff;1=fffff;2=fffff;3=fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/size000066400000000000000000000001601432612301500246140ustar00rootroot00000000000000 L3:0=12582912 L2DATA:0=1310720;1=1310720;2=1310720;3=1310720 L2CODE:0=1310720;1=1310720;2=1310720;3=1310720 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.l2cdp/tasks000066400000000000000000000004161432612301500247730ustar00rootroot000000000000001 2 3 4 6 8 10 11 12 13 14 15 16 17 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 49 50 51 52 53 54 55 56 57 58 59 60 61 62 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 99 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/000077500000000000000000000000001432612301500245315ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/cpus000066400000000000000000000000661432612301500254300ustar00rootroot00000000000000ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/cpus_list000066400000000000000000000000061432612301500264550ustar00rootroot000000000000000-191 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/000077500000000000000000000000001432612301500254645ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/L3_MON/000077500000000000000000000000001432612301500264535ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/L3_MON/max_threshold_occupancy000066400000000000000000000000071432612301500333000ustar00rootroot00000000000000163840 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/L3_MON/mon_features000066400000000000000000000000561432612301500310660ustar00rootroot00000000000000llc_occupancy mbm_total_bytes mbm_local_bytes golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/L3_MON/num_rmids000066400000000000000000000000041432612301500303650ustar00rootroot00000000000000160 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/MB/000077500000000000000000000000001432612301500257625ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/MB/bandwidth_gran000066400000000000000000000000031432612301500306510ustar00rootroot0000000000000010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/MB/delay_linear000066400000000000000000000000021432612301500303250ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/MB/min_bandwidth000066400000000000000000000000031432612301500305050ustar00rootroot0000000000000010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/MB/num_closids000066400000000000000000000000021432612301500302140ustar00rootroot000000000000008 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/info/last_cmd_status000066400000000000000000000000031432612301500305710ustar00rootroot00000000000000ok golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mode000066400000000000000000000000121432612301500253710ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/000077500000000000000000000000001432612301500263135ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_00/000077500000000000000000000000001432612301500277415ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_00/llc_occupancy000066400000000000000000000000111432612301500324720ustar00rootroot0000000000000032440320 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_00/mbm_local_bytes000066400000000000000000000000111432612301500330070ustar00rootroot0000000000000048365568 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_00/mbm_total_bytes000066400000000000000000000000121432612301500330410ustar00rootroot00000000000000264830976 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_01/000077500000000000000000000000001432612301500277425ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_01/llc_occupancy000066400000000000000000000000111432612301500324730ustar00rootroot0000000000000028901376 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_01/mbm_local_bytes000066400000000000000000000000101432612301500330070ustar00rootroot000000000000003342336 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_01/mbm_total_bytes000066400000000000000000000000121432612301500330420ustar00rootroot00000000000000208404480 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_02/000077500000000000000000000000001432612301500277435ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_02/llc_occupancy000066400000000000000000000000111432612301500324740ustar00rootroot0000000000000034406400 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_02/mbm_local_bytes000066400000000000000000000000121432612301500330120ustar00rootroot00000000000000603881472 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_02/mbm_total_bytes000066400000000000000000000000121432612301500330430ustar00rootroot00000000000000974782464 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_03/000077500000000000000000000000001432612301500277445ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_03/llc_occupancy000066400000000000000000000000111432612301500324750ustar00rootroot0000000000000031260672 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_03/mbm_local_bytes000066400000000000000000000000121432612301500330130ustar00rootroot00000000000000693239808 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/mon_data/mon_L3_03/mbm_total_bytes000066400000000000000000000000121432612301500330440ustar00rootroot00000000000000760479744 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/schemata000066400000000000000000000000671432612301500262440ustar00rootroot00000000000000MB:0=4294967295;1=4294967295;2=4294967295;3=4294967295 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/size000066400000000000000000000000761432612301500254310ustar00rootroot00000000000000 L3:0=28835840;1=28835840 MB:0=4294967295;1=4294967295 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3.mbps/tasks000066400000000000000000000004161432612301500256020ustar00rootroot000000000000001 2 3 4 6 8 10 11 12 13 14 15 16 17 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 49 50 51 52 53 54 55 56 57 58 59 60 61 62 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 99 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/000077500000000000000000000000001432612301500235715ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/cpus000066400000000000000000000000661432612301500244700ustar00rootroot00000000000000ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/cpus_list000066400000000000000000000000061432612301500255150ustar00rootroot000000000000000-191 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/000077500000000000000000000000001432612301500245245ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/L3_MON/000077500000000000000000000000001432612301500255135ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/L3_MON/max_threshold_occupancy000066400000000000000000000000061432612301500323370ustar00rootroot0000000000000098304 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/L3_MON/mon_features000066400000000000000000000000561432612301500301260ustar00rootroot00000000000000llc_occupancy mbm_total_bytes mbm_local_bytes golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/L3_MON/num_rmids000066400000000000000000000000041432612301500274250ustar00rootroot00000000000000192 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/MB/000077500000000000000000000000001432612301500250225ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/MB/bandwidth_gran000066400000000000000000000000031432612301500277110ustar00rootroot0000000000000010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/MB/delay_linear000066400000000000000000000000021432612301500273650ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/MB/min_bandwidth000066400000000000000000000000031432612301500275450ustar00rootroot0000000000000010 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/MB/num_closids000066400000000000000000000000021432612301500272540ustar00rootroot000000000000008 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/info/last_cmd_status000066400000000000000000000000031432612301500276310ustar00rootroot00000000000000ok golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mode000066400000000000000000000000121432612301500244310ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/000077500000000000000000000000001432612301500253535ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_00/000077500000000000000000000000001432612301500270015ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_00/llc_occupancy000066400000000000000000000000111432612301500315320ustar00rootroot0000000000000032440320 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_00/mbm_local_bytes000066400000000000000000000000111432612301500320470ustar00rootroot0000000000000048365568 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_00/mbm_total_bytes000066400000000000000000000000121432612301500321010ustar00rootroot00000000000000264830976 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_01/000077500000000000000000000000001432612301500270025ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_01/llc_occupancy000066400000000000000000000000111432612301500315330ustar00rootroot0000000000000028901376 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_01/mbm_local_bytes000066400000000000000000000000101432612301500320470ustar00rootroot000000000000003342336 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_01/mbm_total_bytes000066400000000000000000000000121432612301500321020ustar00rootroot00000000000000208404480 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_02/000077500000000000000000000000001432612301500270035ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_02/llc_occupancy000066400000000000000000000000111432612301500315340ustar00rootroot0000000000000034406400 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_02/mbm_local_bytes000066400000000000000000000000121432612301500320520ustar00rootroot00000000000000603881472 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_02/mbm_total_bytes000066400000000000000000000000121432612301500321030ustar00rootroot00000000000000974782464 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_03/000077500000000000000000000000001432612301500270045ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_03/llc_occupancy000066400000000000000000000000111432612301500315350ustar00rootroot0000000000000031260672 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_03/mbm_local_bytes000066400000000000000000000000121432612301500320530ustar00rootroot00000000000000693239808 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/mon_data/mon_L3_03/mbm_total_bytes000066400000000000000000000000121432612301500321040ustar00rootroot00000000000000760479744 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/schemata000066400000000000000000000000331432612301500252750ustar00rootroot00000000000000MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/size000066400000000000000000000000331432612301500244620ustar00rootroot00000000000000MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nol3/tasks000066400000000000000000000004161432612301500246420ustar00rootroot000000000000001 2 3 4 6 8 10 11 12 13 14 15 16 17 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 49 50 51 52 53 54 55 56 57 58 59 60 61 62 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 99 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/000077500000000000000000000000001432612301500244165ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/cpus000066400000000000000000000000661432612301500253150ustar00rootroot00000000000000ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/cpus_list000066400000000000000000000000061432612301500263420ustar00rootroot000000000000000-191 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/000077500000000000000000000000001432612301500253515ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3CODE/000077500000000000000000000000001432612301500262625ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3CODE/bit_usage000066400000000000000000000001341432612301500301450ustar00rootroot000000000000000=HH000000000SSSSSSSSS;1=HH000000000SSSSSSSSS;2=HH000000000SSSSSSSSS;3=HH000000000SSSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3CODE/cbm_mask000066400000000000000000000000061432612301500277550ustar00rootroot00000000000000fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3CODE/min_cbm_bits000066400000000000000000000000021432612301500306220ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3CODE/num_closids000066400000000000000000000000021432612301500305140ustar00rootroot000000000000008 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3CODE/shareable_bits000066400000000000000000000000061432612301500311500ustar00rootroot00000000000000c0000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3DATA/000077500000000000000000000000001432612301500262615ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3DATA/bit_usage000066400000000000000000000001341432612301500301440ustar00rootroot000000000000000=HH000000000SSSSSSSSS;1=HH000000000SSSSSSSSS;2=HH000000000SSSSSSSSS;3=HH000000000SSSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3DATA/cbm_mask000066400000000000000000000000061432612301500277540ustar00rootroot00000000000000fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3DATA/min_cbm_bits000066400000000000000000000000021432612301500306210ustar00rootroot000000000000001 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3DATA/num_closids000066400000000000000000000000021432612301500305130ustar00rootroot000000000000008 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3DATA/shareable_bits000066400000000000000000000000061432612301500311470ustar00rootroot00000000000000c0000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3_MON/000077500000000000000000000000001432612301500263405ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3_MON/max_threshold_occupancy000066400000000000000000000000071432612301500331650ustar00rootroot00000000000000270336 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3_MON/mon_features000066400000000000000000000000561432612301500307530ustar00rootroot00000000000000llc_occupancy mbm_total_bytes mbm_local_bytes golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/L3_MON/num_rmids000066400000000000000000000000041432612301500302520ustar00rootroot00000000000000176 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/info/last_cmd_status000066400000000000000000000000031432612301500304560ustar00rootroot00000000000000ok golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mode000066400000000000000000000000121432612301500252560ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/000077500000000000000000000000001432612301500262005ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_00/000077500000000000000000000000001432612301500276265ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_00/llc_occupancy000066400000000000000000000000111432612301500323570ustar00rootroot0000000000000032440320 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_00/mbm_local_bytes000066400000000000000000000000111432612301500326740ustar00rootroot0000000000000048365568 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_00/mbm_total_bytes000066400000000000000000000000121432612301500327260ustar00rootroot00000000000000264830976 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_01/000077500000000000000000000000001432612301500276275ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_01/llc_occupancy000066400000000000000000000000111432612301500323600ustar00rootroot0000000000000028901376 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_01/mbm_local_bytes000066400000000000000000000000101432612301500326740ustar00rootroot000000000000003342336 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_01/mbm_total_bytes000066400000000000000000000000121432612301500327270ustar00rootroot00000000000000208404480 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_02/000077500000000000000000000000001432612301500276305ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_02/llc_occupancy000066400000000000000000000000111432612301500323610ustar00rootroot0000000000000034406400 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_02/mbm_local_bytes000066400000000000000000000000121432612301500326770ustar00rootroot00000000000000603881472 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_02/mbm_total_bytes000066400000000000000000000000121432612301500327300ustar00rootroot00000000000000974782464 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_03/000077500000000000000000000000001432612301500276315ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_03/llc_occupancy000066400000000000000000000000111432612301500323620ustar00rootroot0000000000000031260672 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_03/mbm_local_bytes000066400000000000000000000000121432612301500327000ustar00rootroot00000000000000693239808 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/mon_data/mon_L3_03/mbm_total_bytes000066400000000000000000000000121432612301500327310ustar00rootroot00000000000000760479744 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/schemata000066400000000000000000000001161432612301500261240ustar00rootroot00000000000000L3DATA:0=001ff;1=001ff;2=001ff;3=001ff L3CODE:0=001ff;1=001ff;2=001ff;3=001ff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/size000066400000000000000000000000441432612301500253110ustar00rootroot00000000000000L3DATA:0=25952256 L3CODE:0=25952256 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb.cdp/tasks000066400000000000000000000004161432612301500254670ustar00rootroot000000000000001 2 3 4 6 8 10 11 12 13 14 15 16 17 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 49 50 51 52 53 54 55 56 57 58 59 60 61 62 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 99 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/000077500000000000000000000000001432612301500236515ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/cpus000066400000000000000000000000661432612301500245500ustar00rootroot00000000000000ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/cpus_list000066400000000000000000000000061432612301500255750ustar00rootroot000000000000000-191 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/000077500000000000000000000000001432612301500246045ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3/000077500000000000000000000000001432612301500250625ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3/bit_usage000066400000000000000000000001351432612301500267460ustar00rootroot000000000000000=XXSSSSSSSSSSSSSSSSSS;1=XXSSSSSSSSSSSSSSSSSS;2=XXSSSSSSSSSSSSSSSSSS;3=XXSSSSSSSSSSSSSSSSSS golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3/cbm_mask000066400000000000000000000000061432612301500265550ustar00rootroot00000000000000fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3/min_cbm_bits000066400000000000000000000000021432612301500274220ustar00rootroot000000000000002 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3/num_closids000066400000000000000000000000031432612301500273150ustar00rootroot0000000000000016 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3/shareable_bits000066400000000000000000000000061432612301500277500ustar00rootroot00000000000000c0000 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3_MON/000077500000000000000000000000001432612301500255735ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3_MON/max_threshold_occupancy000066400000000000000000000000061432612301500324170ustar00rootroot0000000000000098304 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3_MON/mon_features000066400000000000000000000000561432612301500302060ustar00rootroot00000000000000llc_occupancy mbm_total_bytes mbm_local_bytes golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/L3_MON/num_rmids000066400000000000000000000000041432612301500275050ustar00rootroot00000000000000192 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/info/last_cmd_status000066400000000000000000000000031432612301500277110ustar00rootroot00000000000000ok golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mode000066400000000000000000000000121432612301500245110ustar00rootroot00000000000000shareable golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/000077500000000000000000000000001432612301500254335ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_00/000077500000000000000000000000001432612301500270615ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_00/llc_occupancy000066400000000000000000000000111432612301500316120ustar00rootroot0000000000000032440320 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_00/mbm_local_bytes000066400000000000000000000000111432612301500321270ustar00rootroot0000000000000048365568 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_00/mbm_total_bytes000066400000000000000000000000121432612301500321610ustar00rootroot00000000000000264830976 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_01/000077500000000000000000000000001432612301500270625ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_01/llc_occupancy000066400000000000000000000000111432612301500316130ustar00rootroot0000000000000028901376 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_01/mbm_local_bytes000066400000000000000000000000101432612301500321270ustar00rootroot000000000000003342336 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_01/mbm_total_bytes000066400000000000000000000000121432612301500321620ustar00rootroot00000000000000208404480 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_02/000077500000000000000000000000001432612301500270635ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_02/llc_occupancy000066400000000000000000000000111432612301500316140ustar00rootroot0000000000000034406400 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_02/mbm_local_bytes000066400000000000000000000000121432612301500321320ustar00rootroot00000000000000603881472 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_02/mbm_total_bytes000066400000000000000000000000121432612301500321630ustar00rootroot00000000000000974782464 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_03/000077500000000000000000000000001432612301500270645ustar00rootroot00000000000000golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_03/llc_occupancy000066400000000000000000000000111432612301500316150ustar00rootroot0000000000000031260672 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_03/mbm_local_bytes000066400000000000000000000000121432612301500321330ustar00rootroot00000000000000693239808 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/mon_data/mon_L3_03/mbm_total_bytes000066400000000000000000000000121432612301500321640ustar00rootroot00000000000000760479744 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/schemata000066400000000000000000000000431432612301500253560ustar00rootroot00000000000000L3:0=fffff;1=fffff;2=fffff;3=fffff golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/size000066400000000000000000000001121432612301500245400ustar00rootroot00000000000000L3:0=57671680;1=57671680;2=57671680;3=57671680 MB:0=100;1=100;2=100;3=100 golang-github-intel-goresctrl-0.3.0/test/data/resctrl.nomb/tasks000066400000000000000000000004161432612301500247220ustar00rootroot000000000000001 2 3 4 6 8 10 11 12 13 14 15 16 17 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 49 50 51 52 53 54 55 56 57 58 59 60 61 62 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 99