unity 架构

Hello!  I’m Na’Tosha and I’m the Build and Infrastructure Developer here at Unity Technologies.  While speaking with users at the awesome Unite 2011, I had several people ask me to write a blog post about how Unity Technologies manages to develop, build, and test Unity.

你好！ 我是Na'Tosha，是Unity Technologies的建筑和基础设施开发人员。 在很棒的Unite 2011与用户交谈时，我有几个人要我写一篇关于Unity Technologies如何开发，构建和测试Unity的博客文章。

We do this with a combination of:

我们通过以下方式进行组合：

• Continuous Integration
  持续集成
• Automated Testing
  自动化测试
• Code Hosting
  代码托管
• Code Reviews
  代码评论
• Manual Testing
  手动测试

I’ll talk about how we do the first four of those things here.

我将在这里谈论我们如何做前四件事。

持续集成 (Continuous Integration)

持续集成服务器 (The Continuous Integration Server)

Our builds and automated tests are all run by TeamCity.

我们的构建和自动化测试均由TeamCity运行。

We use TeamCity from JetBrains as our continuous integration solution.  One of the things we like best about TeamCity is that it has a very clear UI that gives the end user a good indication of the state of a project.  For testsuites, it’s easy to see at a glance whether the suite passes or fails and what tests are failing.  It can farm builds out to agents on Windows, OS X, and Linux, and keeps an extensive history of builds and tests, which are very useful when trying to figure out why a test is failing or whether there is something wrong with an agent.  Generally speaking, we are quite happy with the feature-set and also with the support we’ve received from JetBrains.

我们使用JetBrains的 TeamCity作为我们的持续集成解决方案。 我们最喜欢TeamCity的一件事是，它具有非常清晰的UI，可为最终用户很好地指示项目状态。 对于测试套件，可以一目了然地查看套件是否通过或失败以及哪些测试失败。 它可以将构建内容分发给Windows，OS X和Linux上的代理，并保留广泛的构建和测试历史记录，当试图找出测试失败的原因或代理是否存在问题时，这非常有用。 一般而言，我们对功能集以及从JetBrains获得的支持感到非常满意。

Our only real complaint now is regarding the web UI — it can still be slow to load at times (specifically, when loading a large project page that hasn’t been loaded for a while, or when opening the custom build run dialog when a lot of agents are attached).  It can also be difficult regarding managing multiple branch projects, as there is currently no way to synchronize changes between branches that are copies of each other.  However, the projects are stored in XML, and the schema is not complicated, so I have written a tool to do this instead by modifying the XML directly.

现在，我们唯一真正的抱怨是关于Web UI的-有时加载起来仍然很慢(特别是在加载一段时间未加载的大型项目页面时，或者在打开自定义构建运行对话框时的代理人)。 管理多个分支项目也可能很困难，因为当前无法同步作为副本的分支之间的更改。 但是，项目存储在XML中，并且架构并不复杂，因此我编写了一种工具来代替直接修改XML。

建造农场 (The Build Farm)

TeamCity farms builds out to a distributed build farm of between 40 and 50 machines.

TeamCity服务器场可扩展到40到50台计算机的分布式构建服务器场。

Along with TeamCity, we have a build farm of approximately 45 machines, known as build agents.  Most of these are virtual machines running Windows, Mac OS X, and Linux.  We also have a few physical machines used for graphics tests, because testing graphics functionality requires a level of GPU manipulation that we could not achieve with virtual machines.  Our virtual build agents are virtualized with VMWare Workstation on Linux hosts, using a combination of Apple and non-Apple hardware.

与TeamCity一起，我们拥有约45台机器的构建场，这些机器称为构建代理。 其中大多数是运行Windows，Mac OS X和Linux的虚拟机。 我们还拥有一些用于图形测试的物理机，因为测试图形功能需要一定水平的GPU操作，这是虚拟机无法实现的。 我们的虚拟构建代理使用Apple和非Apple硬件的组合在Linux主机上通过VMWare Workstation虚拟化。

The reason for choosing virtual machines, despite the performance hit it gives us, is that virtualization allows us to easily maintain the system — instead of updating 45 machines separately, we update the template virtual machine image for a given operating system, then copy it to the live buildservers.  It also allows us to make every agent appear to be the same hardware-wise, even if they are running on different types of host hardware.  This is important because as we add more build agents, they will be running on different generations of hardware, and for the purposes of testing, we need them to all appear the same.

选择虚拟机的原因是，尽管它给我们带来了性能上的打击，但它是因为虚拟化使我们能够轻松维护系统-无需单独更新45台计算机，而是为给定的操作系统更新模板虚拟机映像，然后将其复制到实时构建服务器。 这也使我们能够使每个代理看起来在硬件方面都是相同的，即使它们在不同类型的主机硬件上运行也是如此。 这一点很重要，因为随着我们添加更多的构建代理，它们将在不同世代的硬件上运行，并且出于测试目的，我们需要它们全部显示相同。

自动化测试 (Automated Testing)

TeamCity provides a very clear interface for seeing the status of test suites.

TeamCity提供了一个非常清晰的界面来查看测试套件的状态。

We have several automated test suites that we run on a regular basis.  These are used, along with manual testing, as a metric for how stable our mainline codebase is, as well as when a feature branch is stable enough to be merged back into trunk.  We currently have four types of test suites:

我们有几个定期运行的自动化测试套件。 这些与手动测试一起用作衡量主线代码库稳定性以及功能分支何时足够稳定以合并回主干的度量。 目前，我们有四种类型的测试套件：

• Unit Tests — these test the correctness of individual functions / small bits of functionality in the code, but they do not test high-level features of the product.  These are used by developers, but not by our continuous integration server.
  单元测试-这些测试用于测试代码中单个功能/少量功能的正确性，但不测试产品的高级功能。 这些由开发人员使用，但不由我们的持续集成服务器使用。
• Graphics Tests — these work by building a Unity project that renders a set of static scenes.  The project is then run on the target platform (desktop, console, or mobile), and screenshots of each scene are saved.  We then compare the saved screenshots against a set of “known good” screenshots.  If the difference is greater than a certain amount, we assert that the test has “failed”.
  图形测试-这些测试通过构建一个Unity项目来工作，该项目渲染一组静态场景。 然后，该项目将在目标平台(桌面，控制台或移动设备)上运行，并保存每个场景的屏幕快照。 然后，我们将保存的屏幕截图与一组“已知良好”的屏幕截图进行比较。 如果差异大于一定数量，则我们认为测试“失败”。
• Functional Tests — these work by launching one instance of the editor or player, and testing various aspects of its functionality.
  功能测试-通过启动编辑器或播放器的一个实例并测试其功能的各个方面来进行这些工作。
• Integration Tests — these are the most high level tests.  They work by launching an instance of the editor or player, testing a particular action or series of actions, and shutting down the editor or player and going on to the next test.
  集成测试-这些是*别的测试。 它们的工作方式是启动编辑器或播放器的实例，测试特定动作或一系列动作，然后关闭编辑器或播放器，然后进行下一个测试。

Our integration tests are executed by our continuous integration server as NUnit tests.  TeamCity can run MSBuild projects directly, and we have a script that runs them with Mono’s Xbuild on Mac OS X.  We developed our own framework for running the graphics tests.

我们的集成测试由我们的连续集成服务器作为NUnit测试执行。 TeamCity可以直接运行MSBuild项目，我们有一个脚本可以在Mac OS X上与Mono的Xbuild一起运行。我们开发了自己的框架来运行图形测试。

Our regression rig tells us if anything has changed in the playback of previously-recorded content.

我们的回归装备会告诉我们先前记录的内容的播放是否发生了任何变化。

We also have a regression rig that we developed ourselves.  The regression rig, similar to the graphics tests, compares content played in Unity against previously recorded content, and checks for regressions.  This is a good way to catch high-level regressions (for example audio having become bit-shifted).

我们还有自己开发的回归模型。 与图形测试类似，该回归装备将在Unity中播放的内容与先前记录的内容进行比较，并检查是否存在回归。 这是捕获高级回归的好方法(例如，音频已移位)。

Our test suites are always being expanded, and, along with our regression rig, they have helped us catch a number of regressions and bugs.

我们的测试套件一直在扩展，并且与我们的回归装备一起，它们帮助我们捕获了许多回归和错误。

代码托管和评论 (Code Hosting and Reviews)

Like many software companies, Unity Technologies has run through a few different version control systems.  In the beginning, there was no version control.  Eventually the first Unity Ninjas began using CVS, and eventually Subversion. About a year ago, we started to investigate distributed version control systems and eventually settled on Mercurial.  Now all of our source code is versioned in Mercurial, except for our public Mono, MonoDevelop, and Boo-related repositories on GitHub.

像许多软件公司一样，Unity Technologies经历了一些不同的版本控制系统。 最初，没有版本控制。 最终，第一个Unity忍者开始使用CVS，最终使用Subversion。 大约一年前，我们开始研究分布式版本控制系统，并最终选择了Mercurial 。 现在，除了GitHub上公开的Mono，MonoDevelop和与Boo相关的存储库之外，我们所有的源代码都在Mercurial中进行了版本控制。

为什么选择水银？ (Why Mercurial?)

Well, we had a few requirements.  The first was that the version control system work with our continuous integration server, which is TeamCity, so we started off considering Git, Mercurial, and Bazaar, and Perforce.  We also wanted something distributed because we have developers working in several different locations, and we also work on multiple platforms.  A distributed system allows our developers around the globe to interact with the remote server less frequently, and it also allows all of us to easily test our changes on multiple local machines without having to share changes that potentially break some other platform with each other.  We also wanted to be able to do feature development in branches and merge them back together successfully.  So we were left with Git, Mercurial, and Bazaar.  We spent some time evaluating these three systems.  We were interested in:

好吧，我们有一些要求。 首先是版本控制系统可与我们的持续集成服务器TeamCity一起使用，因此我们开始考虑Git，Mercurial，Bazaar和Perforce。 我们还希望分发一些东西，因为我们有在不同位置工作的开发人员，而且我们还在多个平台上工作。 分布式系统使我们全球的开发人员可以减少与远程服务器的交互，并且还允许我们所有人轻松地在多台本地计算机上测试更改，而不必共享可能会破坏其他平台的更改。 我们还希望能够在分支机构中进行功能开发并将其成功合并回去。 因此，我们只剩下Git，Mercurial和Bazaar。 我们花了一些时间评估这三个系统。 我们感兴趣的是：

1. A simple, easy to use and understand command-line interface
   一个简单，易于使用和理解的命令行界面
2. Good GUI tools for the system on OS X and Windows
   适用于OS X和Windows上系统的良好GUI工具
3. A good code review tool that works well with the system.
   一个很好的代码检查工具，可以很好地与系统配合使用。

We also wanted a system that we felt had a lot of momentum — an ecosystem that is growing and developing around it.  After a few weeks of testing, we eventually decided on Mercurial because:

我们还想要一个我们觉得有很大动力的系统-一个围绕它成长和发展的生态系统。 经过几周的测试，我们最终决定选择Mercurial，因为：

1. It was substantially more simple to learn and use than Git
   它比Git更加易于学习和使用

2. It had good GUI tools for both Windows (TortoiseHg) and Mac (SourceTree, and now TortoiseHg)

   它具有适用于Windows( TortoiseHg )和Mac( SourceTree以及现在的TortoiseHg )的良好GUI工具。

3. It had a couple of different options for large-scale code review tools
   对于大型代码审查工具，它有几个不同的选择
4. It had a good-sized user-base (which is growing), a regular development cycle, and seemed to be well-adopted by both open-source and and commercial projects
   它拥有庞大的用户群(正在不断增长)，定期的开发周期，并且似乎已被开源项目和商业项目广泛采用

Another very big win for Mercurial is that it was the only DVCS (at the time, at least) that had even attempted to handle the issue of large binaries in the repository.  This was done through a few different publicly-available extensions to Mercurial.  Distributed version control systems, by nature, don’t lend themselves well to codebases with a lot of large binaries, so we knew we would need some system that can cooperate with the version control system to store the large binaries outside of the repository, but still let us version them in some way.  Not having to develop our own system completely from scratch was a really big win.

Mercurial的另一个巨大胜利是，它甚至是唯一的DVCS(至少在当时)试图处理存储库中的大型二进制文件的问题。 这是通过对Mercurial进行的一些不同的公开扩展而完成的。 从本质上讲，分布式版本控制系统不能很好地适应具有大量大型二进制文件的代码库，因此我们知道我们需要一些可以与版本控制系统配合使用的系统，以将大型二进制文件存储在存储库之外，但是仍然让我们以某种方式对其进行版本控制。 不必完全从头开发我们自己的系统是一个巨大的胜利。

如何托管Mercurial以及用于代码审查的内容？ (How to Host Mercurial and What to Use for Code Reviews?)

After we decided on Mercurial, we needed to figure out how to host it and how we would review code.  With this switch, we also wanted to implement a new development policy.  Up until this point, we had always worked with just one central copy of the codebase, except for when we branched for release.  This led to something almost always being broken, which had a serious effect on productivity.  We wanted to be able to easily do feature development in branches; perform peer code reviews, build, and test verification on the branch; then merge it back into the mainline repository once it is complete.  The goal is that trunk is always in a releasable, or nearly releasable, state.

在决定使用Mercurial之后，我们需要弄清楚如何托管它以及如何查看代码。 通过此切换，我们还希望实施新的开发策略。 到现在为止，我们一直只使用代码库的一个*副本，除了分支发布时。 这导致几乎总是发生故障，这严重影响了生产率。 我们希望能够轻松地在分支机构中进行功能开发； 在分支机构执行对等代码审查，构建和测试验证； 然后在完成后将其合并回主线存储库。 目的是使中继线始终处于可释放或几乎可释放的状态。

A Code Review in Kiln.

窑炉中的代码审查。

We looked at a few different code review systems, and eventually settled on Kiln from Fog Creek Software.  Many people already know that we have used FogBugz as our issue tracking system for a while now.  While FogBugz has some room for improvement (specifically, it doesn’t work very well as a public-facing bug tracker), it has done a pretty good job of serving our needs, and we have a really large amount of data in the system.  At this point, we would need a pretty compelling reason to put in the effort to move all of that data to a new system.  Kiln is a Mercurial code hosting system that interfaces with FogBugz and provides nice web-browsing of the repositories, code reviews on a per-changeset basis with one or more reviewers, and a server-side implementation of a Mercurial extension that handles large binaries. We have had some ups and downs with Kiln, mostly with regards to performance.  Our repository size, which is about 1.6 GB with a clean working copy, and has several fairly large binaries, as well as our number of concurrent developers, about 65 and growing, and our build farm, another 45 or so machines, seem to have pushed it to its limits performance-wise.  The self-hosted version of Kiln is currently not built with scalability to large teams (with large repositories) in mind, which results in very slow clones & pushes when there is heavy load. Hopefully, this will get resolved in the future. We’re not sure what our own future is with Kiln, but I will say that its feature set is quite nice, and it has allowed us to move to our desired development model of feature development in branches rather than in mainline.  Beware that it is written in .NET and does not run on Mono, so if you want to consider Kiln, you will need to run a Windows server.  I can say, however, that the Fog Creek support staff has spent countless hours trying to help us work through our various issues with Kiln.

我们研究了几种不同的代码审查系统，最终决定使用Fog Creek Software的 Kiln 。 许多人已经知道我们已经使用FogBugz作为问题跟踪系统已有一段时间了。 尽管FogBugz有一些改进的空间(特别是，它不能作为面向公众的bug跟踪器很好地工作)，但它在满足我们的需求方面做得很好，而且系统中的数据量非常大。 在这一点上，我们将需要一个非常有说服力的理由来努力将所有这些数据移至新系统。 Kiln是一个Mercurial代码托管系统，可与FogBugz交互，并提供很好的Web浏览存储库，基于一个或多个审阅者的每个变更集进行代码审阅以及服务器端实现的Mercurial扩展的实现，可处理大型二进制文件。 我们在窑炉方面经历了一些起伏，主要是关于性能。 我们的存储库大小约为1.6 GB(带有干净的工作副本)，并且具有多个相当大的二进制文件，以及我们的并发开发人员数量(约65个并且正在增长)以及我们的构建场(另外45台左右)在性能方面将其推到了极限。 目前，自托管版本的Kiln并未考虑到大型团队(具有大型存储库)的可伸缩性，这导致重载时克隆和推送速度非常慢。 希望这会在将来解决。 我们不确定Kiln的未来是什么，但是我会说它的功能集非常不错，它使我们能够转移到所需的功能开发模型中，而不是在主线中进行功能开发。 请注意，它是用.NET编写的并且不能在Mono上运行，因此，如果要考虑Kiln，则需要运行Windows服务器。 但是，我可以说，Fog Creek支持人员花费了无数时间来帮助我们解决与窑炉有关的各种问题。

结论 (Conclusion)

Building and testing for so many platforms is a really difficult task — especially as our development team is rapidly growing, and we find ourselves putting strains on infrastructure and processes in areas we didn’t expect.  These tools and processes are fundamental to how the development team gets things done here at Unity.

为如此众多的平台进行构建和测试是一项艰巨的任务-尤其是在我们的开发团队Swift成长的同时，我们发现自己给我们未曾想到的领域的基础架构和流程施加了压力。 这些工具和流程对于开发团队如何在Unity完成工作至关重要。

翻译自: https://blogs.unity3d.com/2011/10/21/build-engineering-and-infrastructure-how-unity-does-it/

unity 架构