Working With Legacy Engineering

Looking back to the past five years, my common experience for joining a project is always looking like this:

Learning business knowledge -> Revealing technical context -> Launching a new project and instilling good practices into the team to make better software as usual.

While it’s not always true for software engineers, I rarely got chance to work on the pre-existing codebase, not to mention a whole legacy engineering.

Things have changed since I stepped on a new journey. My very first challenge is taking over a legacy engineering. I call it legacy engineering because all I have to care is far more than only the codebase, although it is absolutely focal to tackle with the legacy code. The rest of this article will introduce the steps and thoughts I’ve taken during this process.

Communication

A good teacher can at least half the work. Everyone who ever worked in the team can teach and benefit, but the following question is how to learn from people more effectively. My answer is no doubt through good communication.

For those who are interested, I would suggest some great learnings like Nonviolent Communication and Crucial Conversations. No follow-up discussion since we’re not talking about psychology here, but remember it’s indeed the first priority soft skill as a professional.

Documentation

For a hand-over stage, documentation in any form is less effective than a good communication. But it’s worthy to keep valuable knowledge and make it more efficient to spread those to broader stakeholders. Again, it always depends on requirement at the moment for determining the scope of what should be documented. My personal inclination is several topics listed as follows:

  1. Team specific knowledge that everyone should be aware of.
  2. Any confusing/complicated part.
  3. The reason for counter-intuitive decisions made by former team members.

It finally leads to three basic documents for me:

  1. Feature list (Rough is good).
  2. Team engineering practice, like workflow definition and release cycle.
  3. Crucial code design.

Of course, the Evernote in hand to prepare for any unknown technologies and skills.

Codebase

Being familiar with codebase is always the top priority for a newbie, but former topics we just discussed would influence the effectiveness of codebase learning. As software engineering today has adopted many common practices like CI/CD and DevOps, those automation practices can help to guide people along with the code or script, rather than outdated document and lengthy tutorial.

Build (Pipeline)

Build automation is the most matured practice in nowadays software engineering. For a newbie, the build script provides panorama and coarse-grained view of system modularity and dependency, it would be even clearer if the team is maintaining an effective CI pipeline.

A undoubted fact is that the more matured for build automation, the more efficiency for a newbie to get first wave things done. Otherwise, it would take weeks to set up a local environment by referring to an far outdated guide and consulting different team members!

Test

It’s not very common to see “effective tests” in the codebase, not to mention the practice of the whole test pyramid or sandwich. Automation test is a proven practice to assure quality, sometimes it is promoted to be the specification of the codebase. However, bad practice for automation test also does harm other than keeping high quality.

For a newbie, in most cases, the tests especially the bad tests cannot help for specification at all, it shows less value than a good naming practice. My personal experience is that the test is not so good for early learning if it’s not well organized and properly implemented. That’s not saying that it doesn’t play vital role to indicate the consistency when any changes happen.

It’s usually easy for everyone to see the value of automation test, while it also makes this practice be one of the most confusing parts in modern software engineering as lack of deep understanding.

Operation

As DevOps is more adopted, it turns to be more necessary for a newbie to understand operation knowledge. Must-have topics include configuration management and specification of cloud platform and core systems.

Design (Structural) and Refactor

The core of codebase learning is about design. A design could be regarded as a collection of decisions along the software development. For a newbie, it’s more important to concentrate on the crucial design points than knowing every corners.

Although it varies for the different roles, as a backend newbie, the top concern comes into the structural design. To learn software structure,a good starting point could be any selected feature from both business and technical perspective, it can help dive into the deeper codebase to understand the structure, then forms into whole picture of modularity design.

A good approach is to elaborate design with some tools, CRC even Naked CRC is agiler, but UML is also very cool and common, it again depends on team background for these tools selection. More important is that whichever tool is used, only the leading design rules need to be considered, rather than drawing a full system map. During this process, design pattern may benefit but also cause restriction.

It’s not exactly same for design of software structure and logical data model, but the latter could also be learned via any cleaner ways (like E-R diagram), not necessary to dig into physical data model too early unless there is significant reason to do that.

Feature Enhancement

I wouldn’t suggest making big change or refactor on the legacy codebase which is just taken over unless the change is too small to cause any visible influence. Before doing any work alone, a newbie still needs to learn from core team members about:

  1. Architecture Roadmap.
  2. Non-functional requirements.
  3. Technical debts and roadmap of resolution.

Despite the following work would be feature enhancement in most cases. I’d take different strategy depends on the potential influence of the change.

Here I’m not going to discuss the case of a tiny change, that wouldn’t be a big deal anyway. But if the change is going to happen on the higher level design rule but not touch leading yet (which usually implies a broader influence across code), I could try to add new features via extensional way (rather than editing as is code and making feature changes directly, methods metioned in Working effectively with legacy code are good guidance), and extra feature toggles may help for QA and CD.

The worst case is the desired change must happen on leading design rule, which means it won’t work with just simple extension, a big change is definitely required. Even in this case, I wouldn’t edit any as is code in an early stage, and purely technical refactor without guidance from business requirement is also not a good option. Instead, I’d create a separate package or namespace as the neighbor of the code which is going to die, and naming it “experiment” or anything makes sense. Using least effort to make experiment code work as well as automation test, without breaking any as is feature or code. Once new feature and refactor direction is confirmed, the experiment code can be transformed into formal and replace dead one.

The benefit of this “experimental duplication” is that I can continue looking at two implementations and making a trade-off when thinking of the design for following refactor until the new feature is completed and confirmed. Later on, refactor and clean-up can be done to completely remove the deprecation.

Conclusion

In this article, we discussed the strategies I would take while working with legacy engineering. Someone calls it reverse engineering to imply the digestion for an incredibly overdue but complicated legacy codebase or engineering. While in most cases it does not have to make upfront “reverse engineering” to reach critical decision for future, nor do a hasty change on codebase to cause risk. We can certainly take actions to delay that decision making, by mitigating waste and respecting the economic model of our beloved software industry.

Comments

测试三明治和雪鸮探索测试

测试金字塔理论被广泛应用于计划和实施敏捷软件开发所倡导的的测试自动化,并且取得了令人瞩目的成就。本文尝试从产品开发的角度出发,结合Kent Beck最近提出的3X模型和近年来迅速发展的自动化测试技术,提出并讨论一种新的测试层级动态平衡观:三明治模型。同时,为了应对端到端测试在实践中面临的种种挑战,设计并实现了一种面向用户旅程的端到端自动化测试框架——雪鸮。实际项目经验表明,雪鸮能够显著提升端到端测试的可维护性,减少不确定性影响,帮助开发人员更快定位和修复问题,对特定时期的产品开发活动更具吸引力。

背景

测试金字塔

按照自动化测试的层级,从下至上依次为单元测试集成测试端到端测试,尽量保持数量较多的低层单元测试,以及相对较少的高层端到端测试,这就是测试金字塔理论。随着敏捷软件开发的日益普及,测试金字塔逐渐为人所知,进而得到广泛应用。Mike CohnMartin Fowler以及Mike Wacker等先后对测试金字塔进行了很好的诠释和发展,其主要观点如下:

  • 测试层级越高,运行效率就越低,进而延缓持续交付的构建-反馈循环。
  • 测试层级越高,开发复杂度就越高,如果团队能力受限,交付进度就会受到影响。
  • 端到端测试更容易遇到测试结果的不确定性问题,按照Martin Fowler的说法,这种结果不确定性的测试毫无意义。
  • 测试层级越低,测试的代码隔离性越强,越能帮助开发人员快速定位和修复问题。

3X模型

2016年起,敏捷和TDD先驱Kent Beck开始在个人facebook主页撰写系列文章,阐述产品开发的三个阶段——Explore、Expand和Extract,以及在不同阶段中产品与工程实践之间的关系问题,即3X模型。近二十年软硬件技术的飞速发展,使得软件开发活动面临敏捷早期从未遇到的市场变革,而根据在facebook工作的经历,Kent Beck把产品开发总结为三个阶段:

  • 探索(Explore),此时的产品开发仍处于非常初期的阶段,仍然需要花费大量时间寻找产品和市场的适配点,收益也是最低的阶段。
  • 扩张(Expand),一旦产品拥有助推器(通常意味着已经找到了市场的适配点),市场需求就会呈现指数级上升,产品本身也需要具备足够的伸缩性以满足这些需求,由此收益也会快速上升。
  • 提取(Extract),当位于该阶段时,公司通常希望最大化产品收益。但此时收益的增幅会小于扩张阶段。

3X

Kent Beck认为,如果以产品是否成功作为衡量依据,那么引入自动化测试在探索阶段的作用就不大,甚至会延缓产品接受市场反馈循环的速度,对产品的最终成功毫无用处,还不如不引入;当位于扩张阶段时,市场一方面要求产品更高的伸缩性,另一方面也开始要求产品保证一致的行为(例如质量需求),那么此时就该引入自动化测试来保证产品的行为一致性;当产品最终处于提取阶段时,任何改动都应以不牺牲现有行为为前提,否则由此引发的损失可能远高于改动带来的收益,此时自动化测试就扮演了非常重要的角色。

测试工具爆炸式增长和综合技能学习曲线陡升

根据SoftwareQATest网站的历史数据,2010年记录的测试工具有440个,共划分为12个大类。这个数字到2017年已经变为560个,共15个大类,且其中有340个在2010年之后才出现。也就是说,平均每年就有50个新的测试工具诞生。

面对测试工具的爆炸式增长,一方面所支持的测试类型更加完善,更加有利于在产品开发过程中保证产品的一致性;另一方面也导致针对多种测试工具组合的综合技能学习曲线不断上升。在实践中,团队也往往对如何定义相关测试的覆盖范围感到不知所措,难以真正发挥测试工具的效用,也很难对产品最终成功作出应有的贡献。

从金字塔到三明治

作为敏捷在特定时期的产物,测试金字塔并不失其合理性,甚至还对自动化测试起到了重要推广作用。但是,随着行业整体技术能力的不断提升,市场需求和竞争日趋激烈,在项目中具体实施测试金字塔时往往遭遇困难,即便借助外力强推,其质量和效果也难以度量。

此外,随着软件设计和开发技术的不断发展,低层单元测试的传统测试技术和落地,因前、后端技术栈的多样化而大相径庭;同时,在经历过覆盖率之争,如何确保单元测试的规范和有效,也成为工程质量管理的一大挑战;高层的端到端测试则基本不受技术栈频繁更替的影响,随着不同载体上driver类技术的不断成熟,其开发复杂度反而逐渐降低。

这里讨论一种新的测试层级分配策略,我们称之为三明治模型 。如下图所示,该模型允许对不同测试层级的占比进行动态调整,说明了倒金字塔形、沙漏形以及金字塔形分配对特定产品开发阶段的积极作用。

Sandwich

产品开发的自动化测试策略

根据3X模型,在探索初期往往选择避开自动化测试。一旦进入扩张期,产品的可伸缩性和行为一致性就成为共同目标,但此时也常会发生大的代码重构甚至重写,如果沿用测试金字塔,无论补充缺失的单元测试,还是只对新模块写单元测试,都既损害了产品的快速伸缩能力,也无法保证面向用户的产品行为一致性。因此,如果在探索后期先引入高层的端到端测试,覆盖主要用户旅程,那么扩张期内所产生的一系列改动都能够受到端到端测试的保障。

需要注意的是,用户旅程在产品即将结束探索期时通常会趋于稳定,在扩张期出现颠覆性变化的概率会逐渐减少,端到端测试的增长率会逐步下降。

除此以外,随着扩张期内不断产生的模块重构和服务化,团队还应增加单元测试和集成测试的占比。其中,单元测试应确保覆盖分支场景(可以在CI中引入基于模块的覆盖率检测);集成测试和某些团队实践的验收测试,则需进一步覆盖集成条件和验收条件(在story sign-off和code review时验收)。

许多新兴的测试技术和工具擅长各自场景下的验收测试,但更重要的仍是识别产品阶段和当前需求,以满足收益最大化。

Sandwich-3x

由此我们认为,随着产品开发的演进,测试层级的分配应参考三明治模型,动态调整层级占比,更加重视运营和市场反馈,致力于真正帮助产品走向成功。

端到端测试的机遇和挑战

与其他测试层级相比,端到端测试技术的发展程度相对滞后。一方面,作为其基础的driver工具要在相应载体成熟一段时间之后才能趋于稳定,web、mobile无不如是。另一方面,端到端测试偏向黑盒测试,更加侧重描述用户交互和业务功能,寻求硬核技术突破的难度较高,于是较少受开发人员青睐。但是,由于端到端测试更接近真实用户,其在特定产品开发活动中的性价比较高,有一定的发展潜力。

然而,当前实践中的端到端测试,普遍存在如下问题:

  • 低可维护性。一般实践并不对测试代码质量作特别要求,而这点在端到端测试就体现得更糟。因为其涉及数据、载体、交互、功能、参照(oracle)等远比单元测试复杂的broad stack。虽然也有Page Object等模式的广泛应用,但仍难以应对快速变化。
  • 低运行效率。如果拿单次端到端测试与单元测试相比,前者的运行效率肯定更低。因此只一味增加端到端测试肯定会损害构建-反馈循环,进而影响持续交付。
  • 高不确定性。同样因为broad stack的问题,端到端测试有更高的几率产生不确定测试,表现为测试结果呈随机性成功/失败,进一步降低运行效率,使得真正的问题很容易被掩盖,团队也逐渐丧失对端到端测试的信心。
  • 难以定位问题根因。端到端测试结果很难触及代码级别的错误,这就需要额外人工恢复测试环境并尝试进行问题重现。其中所涉及的数据重建、用户交互等会耗费可观的成本。

方法

为了解决传统端到端测试遇到的种种挑战,本文设计了一种面向用户旅程的端到端自动化测试框架——雪鸮(snowy_owl),通过用户旅程优先、数据分离、业务复用和状态持久化等方法,显著提高了端到端测试的可维护性,降低不确定性的影响,并且能够帮助团队成员快速定位问题。

用户旅程驱动

端到端测试应尽量贴近用户,从用户旅程出发能保证这一点。在雪鸮中,用户旅程使用被称作play books的若干yaml格式的文件进行组织,例如下列目录结构:

play_books/
  core_journey.yml
  external_integration.yml
  online_payment.yml

其中每个play book由若干plots所组成,plot用于表示用户旅程中的“情节”单位,其基本特征如下:

  • 单一plot可以作为端到端测试独立运行,例如发送一条tweet的情节:
1
2
3
4
5
6
7
8
9
10
11
12
SnowyOwl::Plots.write 'send a plain text tweet' do
  visit '/login'
  page.fill_in 'username', with: 'username'
  page.fill_in 'password', with: 'password'
  page.find('a', text: 'Sign In').click
  # verify already login?
  page.find('a', text: 'Home').click
  # verify already on home page?
  page.fill_in 'textarea', with: 'Hello World'
  page.find('a', text: 'Send').click
  # verify already sent?
end
  • 单一plot应是紧密关联的一组用户交互,并且具备体现一个较小业务价值的测试参照。
  • plot可以被play book引用任意次从而组成用户旅程,play book同时定义了所引用plots之间的顺序关系,基本语法如下所示:
1
2
3
4
---
- plot_name: send a plain text tweet
  digest: 2aae6c35c94fcfb415dbe95f408b9ce91ee846ed
  parent: d6b0d82cea4269b51572b8fab43adcee9fc3cf9a

其中plot_name表示情节的标题,digest和parent分别表示当前情节引用在整个端到端测试过程中的唯一标识和前序情节标识,初期开发人员可以通过各个情节的引用顺序定义用户旅程,大多数情况下digest和parent将由系统自动生成并维护。

整个play books集合将是一个以plots为基础组成的森林结构,而端到端测试的执行顺序则是针对其中每棵树进行深度遍历。

通用业务复用

由于plot本身必须是一个独立可运行的端到端测试,那么plots之间通常会共享一部分交互操作,例如用户登录。雪鸮允许把高度可复用的交互代码进行二次抽取,称作determination:

1
2
3
4
5
6
7
8
SnowyOwl::Determinations.determine('user login') do |name, user_profile|
  # return if already login
  visit '/login'
  page.fill_in 'username', with: user_profile[:username]
  page.fill_in 'password', with: user_profile[:password]
  page.find('a', text: 'Sign In').click
  # verify already login?
end

这样,plot的代码就可以简化成:

1
2
3
4
5
6
7
8
SnowyOwl::Plots.write 'send a plain text tweet' do
  determine_user_login({username: 'username', password: 'password'})
  page.find('a', text: 'Home').click
  # verify already on home page?
  page.fill_in 'textarea', with: 'Hello World'
  page.find('a', text: 'Send').click
  # verify already sent?
end

这里应注意Determination和Page Object的区别。看似使用Page Object可以达到相同的目的,但是后者与Page这一概念强绑定。而Determination更加侧重描述业务本身,更符合对用户旅程的描述,因此比Page Object在plot中更具适用性。当然,在描述更低层的组件交互时,Page Object仍然是最佳选择。

测试数据分离

合理的数据设计对描绘用户旅程非常重要,雪鸮对测试逻辑和数据进行了进一步分离。例如用户基本数据(profile),同样是使用yaml文件进行表示:

data/
  tweets/
    plain_text.yml
  users/
    plain_user.yml

那么在plot的实现中,就可以使用同名对象方法替代字面值:

1
2
3
4
5
6
7
8
SnowyOwl::Plots.write 'send a plain text tweet' do
  determine_user_login({username: plain_user.username, password: plain_user.password})
  page.find('a', text: 'Home').click
  # verify already on home page?
  page.fill_in 'textarea', with: plain_text.value
  page.find('a', text: 'Send').click
  # verify already sent?
end

情节状态持久化

雪鸮的另一个重要功能是情节状态的持久化和场景复原。为了启用情节状态持久化,开发人员需要自己实现一个持久化脚本,例如对当前数据库进行dump,并按照雪鸮提供的持久化接口把dump文件存储至指定位置。

当端到端测试运行每进入一个新的情节之前,系统会自动执行持久化脚本。也就是说,雪鸮支持保存每个情节的前置运行状态。

当端到端测试需要从特定的情节重新开始运行时,雪鸮同样会提供一个恢复接口,通过用户自定义的数据恢复脚本把指定位置的dump文件恢复至当前系统。

该功能有两处消费场景:

  • 由于broad stack的问题,端到端测试不确定性的技术因素一般较为复杂。实际经验表明,测试的随机失败率越低,就越难以定位和修复问题,而通过不断改进测试代码的方式消除这种不确定性的成本较高,效果也不好。但是,可以尽量消除不确定性带来的影响。例如,不确定测试导致的测试失败,通常会导致额外人工验证时间,完全可以选择让系统自动重试失败的测试。另一方面,重试会造成测试运行效率降低,特别是针对端到端测试。当一轮端到端测试结束后,雪鸮只会自动重试失败的情节测试,同时利用该情节对应的数据dump文件保证场景一致性,这就减少了重试整个端到端测试带来的运行效率下降问题。
  • 当团队成员发现端到端测试失败,通常需要在本地复现该问题。而借助测试dump文件,可以直接运行指定plot测试,从而避免额外的人工设置数据和交互操作,加快问题定位和解决。

实践

雪鸮在笔者所在的项目有超过6个月的应用时间。该项目在产品开发方面长期陷入困境,例如过程中同时兼具了3X每个阶段的特点,不仅缺少清晰的产品主线,还背负了接棒遗留系统的包袱。这种状况对工程质量管理提出了更大挑战。

项目采用雪鸮对已有端到端测试进行了重构,生成了一个核心用户旅程和三个涉及外部系统集成的重要用户旅程,包含24个plots,9个determinations,使端到端测试实现了长期稳定运行。在本地相同软硬件环境下,不确定性导致的随机失败从原有10%降低至1%以内,部署至云环境并采用headless模式后,连续15天测试失败记录为零,运行效率的损失可以忽略不计。同时,当用户旅程产生新分支时,可以引入新的情节测试节点,并且根据业务需求将其加入现有play book树,从而实现端到端测试的快速维护。

持续集成与常态化运行

项目完整的端到端测试的平均运行时间保持在19分钟左右,为了不影响现有持续集成节奏,CI每30分钟自动更新代码并运行端到端测试,结果在dashboard同步显示,一旦发生测试失败,第一优先级查找失败原因并尝试在本地复现和修复。

常态化运行端到端测试的另一个好处是,能够以低成本的方式实现24小时监控系统各个组件的功能正确性,有助于更早发现问题:一次,产品即将上线的支付功能发生异常,查看CI记录发现端到端测试在晚上9:15左右出现了首次告警。通过及时沟通,确认是海外团队在当时擅自改动了支付网关的一个配置,造成服务不可用的问题,并迅速得以解决。

结论与展望

Kent Beck的3X模型,提出了从不同产品开发阶段看待工程实践的新视角。而敏捷一贯推崇的TDD等实践,更多体现在个人技术专长(Expertise)方面,与产品是否成功并无必然联系。然而,程序员的专业主义(Professionalism)的确同时涵盖了技术专长和产品成功两个方面,二者相辅相成。因此,如何通过平衡众多因素并最终提高整体专业性,这才是软件工程面临的经典问题。本文给出的测试三明治模型,目的就是帮助思考产品开发过程中测试层级间的平衡问题。

为了应对现有端到端测试面临的挑战,本文设计并实现了一种新的面向用户旅程的端到端测试框架,通过职责隔离、业务复用和状态持久化等手段,构建了易于维护且更加有效的端到端测试。同时,基于上述方法构建的测试代码,更易于和自动化测试的其他研究领域相结合,在诸如测试数据构建、用例生成、随机测试和测试参照增强等方向有进一步的应用潜力。

Comments

微情境混合现实

从前有一池清潭,在附近的居民眼里它和普通池塘并没有什么区别。某天一位富绅来到池边,搬起一块巨石砸进了池中心,泛起的沉渣顿时把池水变成了浊潭。富绅见人就说,突然变浑浊的潭水里其实是出土了宝藏,价值万贯。消息传开,好奇的人们不远万里赶到潭边,有的确实想发笔横财,有的只是为了观光。富绅自己则在潭边围起一段栅栏,向来访者销售门票,不日累积万贯。不久,别处的清潭都变浊了。

近年火爆的VR/AR在大众眼中是“黑科技”,其实早在50年前(The Sword of Damocles,1965)就被人玩剩了,他的创作者是图形学界祖师之一Ivan Sutherland。但可能是发现自己已经走得太远(实际上当时现代图形学基础还没有完全确立),于是萨大爷就带着他一票学生和下属回去后方铺路,后者们先后发明了Smalltalk编程语言、Gouraud着色算法、图形反走样算法,创办创意设计巨头Adobe以及高性能计算鼻祖SGI,甚至还联合乔帮主创办了Pixar(现任Pixar/Disney动画总裁的Edwin Catmull也是图形学巨擘,发明了灰常实用的Catmull-Rom样条函数,直到现在还被用于关键帧插值和渲染头发丝)。

有意思的是,VR/AR在理论上的研究早已从感知器官上升到哲学范畴……多年来在实践上却仍然停留在科技馆和特殊行业应用领域。最终,geek们期待的科技大跃进并没有真正到来,尘埃褪去,呈现的却是更加亲民的头显(HMD)和各种奇思妙想的控制器(Controller),无不体现着以人为本的时代理念。而如果你能尝试SteamVR上的应用、特别是体验下Proto奖的获奖作品时,你会欣慰地发现,原来真正有梦想、有创造力的人还在不断涌现,你就会从各种捞钱、炫富的纸金现实中暂时解脱出来了。

如果避开硬科幻,从终极关怀的角度审视VR/AR,我们所处时代的主题原来并非“黑科技”,而是“低门槛”和“大众化”。这里,本文提出一种特定场景下的混合现实产品思路,最后以一个真实的开源Demo为例,希望能为许多有想法的人打开一扇窗,鼓励更多人能参与到这一领域中来。

C22-Full

为什么是特定场景?

门槛是首要考虑因素。以Daydream为例,在如今诸多VR/AR平台中,Daydream的优势是非常易见的:它太容易接近大众了。对于其它竞品平台,则更多聚焦在高端和行业应用上,不利于小团队做快速原型开发和内容创新(就是说只适合憋大招)。

而Daydream的劣势目前也很明显:3-DOF的控制器无法捕捉用户的全身动作,反馈缺失导致了沉浸感不足(Vive的这一优势在市面上目前还无人匹敌);另外,移动设备受限的性能、参差不齐的规格也导致潜在的用户体验问题,这恐怕也是Google率先携Pixel发布的初衷。

因此在设计VR/AR产品时,需要首先考虑平台对应用场景的限制。

微情境(Micro-Situation)

理论上说,VR/AR技术几乎可以被用于任何情境。基于现实考量,在“微情境”中实现的性价比则更高。

例如非厨师职业学习做饭,传统上可能是一对一的学徒体系;后来发展成书籍报刊、电视传播。前者属于文字模式更多得靠个人天赋,后者的声音+图像模式就更容易被大众所接受。

但有人可能会质疑为什么不真的去尝试做一顿饭?其实这相当于混淆了学习和实践两种活动,“学做饭”和“做饭”本身不是一回事(当然你也可以说在实践中学习,但你也不能证明你是在完全未经过学习的情况下做了一顿饭,况且如果真的是这样,那么满足各位食客的可能性恐怕微乎其微)。

再来看“虚拟学做饭”这一活动的价值,最大的劣势就是你无法从外观之外获得任何的有效反馈,包括直接吃掉自己的学习成果。而优势是节省钱、食材、时间,切身经历每一道工序,这是现有的学习形式都不具备的。

从这一情境本身来说,用户只需要一间整洁的厨房,摆放整齐的食材和调味品,再加上面前一座灶台即可。而交互内容仅需要双手参与。那么即将进入寻常百姓家的Daydream平台完全满足需求,于是皆大欢喜。

其实这里需要考虑的重点在“微”上,什么样的情境才算是“微情境”呢?我们不尝试进一步去做更多的文字游戏,对于“微情境”的发现要更多依赖对情境本身、以及对VR/AR技术的理解和运用,这也是不可避免的过程。

这里再延伸到一个稍“大”一些的微情境:产品说明书

相信大家对各式各样的产品说明书都不陌生,但从现实的生活经验来看,在大多数情况下,要么产品自身完全做到了自说明,要么很难快速(或者根本无法)从说明书中找到自己需要的答案。多年来,尽管平面设计不断进步,多媒体新招层出不穷,也很难断言一个真正有效的用户-产品沟通方式,而VR/AR的普及将为此带来全新的维度。

例如你网购了一套书柜,然而需要自己动手安装,原本自信满满的你花了很长时间,终于无奈地瘫坐在沉甸甸的零件旁,边满头大汗地研究起貌似手绘的安装说明……在当前情境中,家具的零件都可以被虚拟化,用户也不需要真的在场景中走来走去,而是直接上手了解零件功能和安装步骤即可——可以想见,当平台越廉价,“微情境”中的VR/AR应用所带来的附加价值就会越高(例如直接嵌入产品官网Web,并借助移动设备进行VR/AR显示,具体实现请参考文末Demo)。

或许,人们会在不久之后的春晚上体验到VR/AR抢红包之类的全民娱乐呢?

为什么是混合现实(MR)?

在这波VR行情之前,就有许多AR应用运行在智能手机上,此外还包括不久前的google glass,却始终没有杀手级应用出现(酝酿多年也才蹦出一只Pokemon Go…)。原因甚多,其中市场太小、远离大众是一个非常重要的因素。难道VR就没有市场的疑问?否则巨头们为何集体押注在后者身上?由此可猜想投资VR的回报率明显要更高一筹,却万不能沉迷于只做游客。

值得一提的是Hololens展示了当前无与伦比的MR技术,无奈距离大众太过遥远,注定只会面向专业领域。实际上基于现有的VR设备要想直接实现接近Hololens的MR并不困难,毕竟只需要一个透明显示和空间位置标定,如果要做进一步的场景感知会稍复杂(但也已经不是壁垒),不过由此发展的成本肯定比一步到位要低许多。

(这里不得不说句题外话,平台大战可以造就繁荣的假象,但也可能扰乱了内容创造者们的思想,从而延缓技术的最终普及,这种风险在当前看来是不可避免的)

然而如果单纯比拼想象力,MR更胜VR/AR,虽然前者只是模糊了后者之间的界限。在接受程度上,人们总会倾向于那些更加抽象、简洁且表现力更强的形式,MR无疑将会最终胜出。

真正的困难之处在于,MR不仅仅在于两幅人眼画面的结合,也意味着更加复杂的应用场景,以及更具挑战的交互模式,而不是一个看起来更高级的HMD。

来自22世纪的程序员(C22)

C22是一个基于传统PC+Cardboard的微情境应用。它展示了未来的程序员,能够摆脱固定显示器的束缚,使用HMD、键鼠就可以编写代码,并且在自身环境周围展示各种信息的技术。

C22-Right

如上图所示,PC中的应用信息被独立传送至HMD中创建的每个虚拟屏幕上,程序员可以在三维场景中浏览各种信息。

除了信息浏览外,C22允许程序员在盲打的情况下使用键盘、鼠标在对应的应用上进行操作,并且能够实时同步。

由于当前C22无法显示在一块透明质的HMD上(类似Hololens),因此也无法真正让普通用户感受到在现实中操作应用。此外由于分辨率的限制,C22目前能实现的用户体验也比较受限

C22-Left

为了快速实现产品原型,C22全部使用JavaScript编写,并且采用了许多尚未广泛兼容的技术(详见项目github主页),由此带来的好处是核心部分搭建只用了两个晚上的时间,后期为了提高体验可能会进一步扩展当前技术栈。

同样由于兼容问题,C22目前只支持Android设备,并计划在2017年第一季度支持Daydream,届时,用户可以在周围自由设置信息布局。并且由于WebVR将得到浏览器原生支持,C22的渲染能力也将得到显著提升。

2015年举行的Proto颁奖大会上,年近八旬的萨大爷获虚拟现实“创始人奖”,大神激励台下一帮天才后辈说:“内容意味着一切,摄像机(指技术)并不创造内容,只有最伟大的创造者才得以使用技术把现实带给人们”。

闻者深受共鸣。

mr
Comments

Metaprogramming Ruby: Core Concepts - Extensions

As we already know, ruby provides several ways to extend our classes and objects including but not limit to OO inheritance, Module mix-in, Dynamic Methods, Monkey patching / Refinement, Eval family, Ghost methods, Callable objects, Singleton classes / methods, etc. But there are still some important tips deserve to be mentioned, before we moving forward.

1. Include vs Extend

Firstly let’s see an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
module Foo
  def self.foo
    p 'Hello world'
  end

  def say_hi
    p 'Hello world'
  end
end

class Bar
  include Foo
end

Bar.foo #NoMethodError: undefined method `foo' for Bar:Class
Bar.new.say_hi #Hello world

It seems that module mix-in can only involve instance methods but not others like class methods. But how to do similar thing at class level? You must hear about extend:

1
2
3
4
5
6
7
8
9
10
11
module Foo
  def foo
    p 'Hello world'
  end
end

class Bar
  extend Foo
end

Bar.foo #Hello world

Extend is used in object level, so we are sure it can also be used for any class in ruby. But in previous article, we know that class method is actually saved as a singleton method of the original class, also instance method of its singleton class. So that should also happen on include:

1
2
3
4
5
6
7
8
9
10
11
12
13
module Foo
  def foo
    p 'Hello world'
  end
end

class Bar
  class << self
    include Foo
  end
end

Bar.foo #Hello world

Thus we can regard extend as a kind of special usage on include through above examples.

2. Method wrappers

Method wrapper means wrapping an existing method inside a new method, which is very useful when you want to make extension without changing the source, like code in standard library or other cases.

There are several ways to implement method wrappers in ruby, and they are all in composite form of primitives which’ve already been introduced in previous articles. We’ll go through below.

Around Alias

Module#alias_method (also the key word ‘alias’) is used to give another name to ruby methods. It will involve more accessibility if an usual method could have different domain names(e.g. size, length and count). Also more flexibilities if you want, like code below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
class Foo
  def say_hi
    p 'Hello'
  end
end

class Foo
  alias_method :say_hello, :say_hi

  def say_hi
    say_hello
    p 'World'
  end
end

foo = Foo.new
foo.say_hi #Hello\nWorld\n

This is just a kind of wrapper using open class, alias_method, and method redefinition.

Refinement

We talked about refinement and suggested using refine instead of monkey patch. Actually refinement is even more powerful than that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
class Foo
  def say_hi
    p 'Hello'
  end
end

module FooRefinement
  refine Foo do
    def say_hi
      super
      p 'World'
    end
  end
end

using FooRefinement

Foo.new.say_hi #Hello\nWorld\n

Only thing you need to notice is that the key word ‘using’ may not work well with your IRB environment, which means you couldn’t get result in mind for some versions of ruby if you run those code in IRB. See more information here.

The benefit of refinement wrapper is controllable scope of wrapper unlike around alias which affects globally. However, accessibility to original method is also lower than alias way.

Prepending

Module#prepend is the simplest way to implement method wrapper without scope configurability like refinement. But much more clear than other two.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
class Foo
  def say_hi
    p 'Hello'
  end
end

module FooPrepending
  def say_hi
    super
    p 'World'
  end
end

Foo.class_eval do
  prepend FooPrepending
end

Foo.new.say_hi #Hello\nWorld\n

3. Class Macros

Ruby objects have no attributes - May this won’t surprise or confuse you too much. Indeed we’ve hear about instance variables or class variables, but you can not access them directly from outside. That means getter or writer can not be avoided:

1
2
3
4
5
6
7
8
9
10
11
12
13
class Foo
  def bar=(value)
    @bar = value
  end

  def bar
    @bar
  end
end

foo = Foo.new
foo.bar = 'Hello'
foo.bar #Hello

It’s just not ruby style! Ruby provides series accessors for class definition using metaprogramming api, like attr_accessor, attr_reader and attr_writer, they are quite intuitive to use:

1
2
3
4
5
6
7
class Foo
  attr_accessor :bar
end

foo = Foo.new
foo.bar = 'Hello'
foo.bar #Hello

attr_* come from Module as private instance methods, thus they all can be used in module or class definitions.

Comments

Metaprogramming Ruby: Core Concepts - Eval Family II

In previous article several basic scope controlling tools are introduced, like block, proc, lambda, and method object. All of them should work well when you organize them in your code reasonably. There are more flexible way to mix code and bindings in ruby - eval, which may not be used broadly than others, so that considerable traps about it are still unclear for us.

3. instance_eval / instance_exec

BasicObject#instance_eval can be used almost everywhere in ruby, you can use it like this:

1
2
3
foo = Object.new
foo.instance_eval { @bar = 'hello' }
foo.instance_variable_get('@bar') #hello

In the block of instance_eval, the scope changes to the instance foo, thus any operations inner it should bind to the instance, except closure variables.

BasicObject#instance_exec has similar feature to eval one, but with arguments support. This benefit shows as below:

1
2
3
4
5
6
7
8
9
class Foo
  def initialize
    @bar = 'hello'
  end
end
output = 'world'
foo = Foo.new
foo.instance_exec(output){|output| output = "#{@bar} #{output}"}
output #world

4. class_eval / class_exec

Module#class_eval works for evaluating a block in the context of an existing class. It seems to be similar to instance_eval which will have scope changes to instance self, but also changes current class (excluding singleton class).

class_eval can be used to open a existing class, without using keyword ‘class’. That means it has no scope changing issue compared with keyword way. Actually it even does not need to know the constant name of the target, while keyword way indeed needs.

Also Module#class_exec plays same role like instance_exec for instance_eval.

Think about the thing what we’ve discussed several weeks ago, in ruby all items are instance of class, including classes no matter internal or customized. Which means instance_eval and class_eval both can work for classes in many cases, but indeed have different self meaning.

You may also notice that there is also Module#module_eval / Module#module_exec methods, but they are just alias for class_eval / class_exec without any other changes.

5. Kernel#eval

Unlike instance_eval/class_eval, Kernel#eval only accept a string of ruby code as its argument, run the code and give the result. Even it’s so powerful, using string of code is not a good choice if you have other ways. The most issue for string of code is security.

Suppose you have eval expression in your code, which means others can evaluate any code using such method, including get/set operations. Ruby defines safe levels, which actually limits the evaluation of code from outside. By default, ruby will mark potentially unsafe objects which many come from external sources. And user could config a global variable $SAFE, it’s range from 0 to 3(0 by default) with more strict.

1
2
3
$SAFE = 1
eval "p 'hello #{gets()}'"
world #SecurityError: Insecure operation - eval

By default, eval can accept almost any code from outside. However, it will not be permitted for code from I/O device(tainted object) if $SAFE is none zero. Below gives more details about $SAFE:

$SAFE Constraints
0 No checking of the use of externally supplied (tainted) data is performed. This is Ruby’s default mode.
>= 1 Ruby disallows the use of tainted data by potentially dangerous operations.
>= 2 Ruby prohibits the loading of program files from globally writable locations.
>= 3 All newly created objects are considered tainted.
Comments

Metaprogramming Ruby: Core Concepts - Eval Family I

In ruby, block is the cornerstone of metaprogramming with a surprisingly powerful scope controlling capability. Based on this definition, much brilliant interfaces are introduced, and help to implement many valuable features, like callable objects, eval and its huge family.

1. blocks

Block represents a snippet of code, it permits to be called immediately or lately depends on different cases. But compared with some similar concepts in functional programming languages, like Lisp, block still has more limits, and more readability respectively.

In commonly, block can be used like this:

1
2
3
4
5
6
7
8
9
10
11
def show_hello
  yield 'hello' if block_given?
end

show_hello { |say| puts say }

show_hello do |say|
  file = File.open('/tmp/output.txt', 'w')
  file.write say
  file.close
end

‘yield’ is a ruby keyword used to call blocks sent to the current method. block_given? is an instance method from Kernel to probe whether there is a block for current method.

One of the most useful aspect of block is closure, it captures bindings where it’s defined, and avoid impact of scope changing by connecting to those bindings, like flat scopes, and shared scopes.

Frankly speaking, block is actually not responsible for running code, but only representation (Except yield, which indeed means running code immediately). There are more powerful tools to help enhance block usage.

2. callable objects

Block is just like a package of code, and you need to use yield to execute it if you like. However, there are more ways to package code in ruby, including Proc and lambda.

Proc

We already know that block is not an object in ruby which is really quite a few, but Proc is basically a block turned object, can be seen as a consistent form of block, and you do not have to use yield to run it immediately, it will be running later as you want (Deferred Evaluation). you can define a Proc like this:

1
2
3
4
inc = Proc.new {|x| x + 1}
inc.call(1) #2
dec = proc {|x| x - 1}
inc.call(2) #1

lambda

Except for Proc, lambda can also be used for transferring blocks, but with simpler and a little different way:

1
2
3
4
inc = lambda {|x| x + 1}
inc.call(1) #2
dec = ->(x) {x - 1}
dec.call(2) #1

& operator

Once you have defined a block for a method, there is a way to convert it to a Proc inner method by using &. For example:

1
2
3
4
5
6
7
8
9
10
11
12
def bar
  yield 'ruby'
end

def foo(&operation)
  bar(&operation)
  operation.call('world')
end

foo {|x| puts "hello #{x}"}
# hello ruby
# hello world

& can be seen like transferring a block into Proc, but you need to remove & if you want to use it as a Proc.

Comparison between procs and lambdas

You may notice that lambda is also a Proc, but you can still use Proc#lambda? to see the actual type of the target. Besides there are two important differences: keyword ‘return’ and arguments checking.

a. return

As plain blocks, the program will return out of scope where block defined if return statement is set inner block, so does Proc (which may mean that the return may cause exception when the block is called in nonreturnable scope, like top-level). While for lambda, the return only runs out of block, not even farther.

b. arity

Arity means the number of arguments. In real case, lambda has less tolerance than Proc, which means that lambda requires correct number of arity, which is no need for procs.

Method objects

All methods are objects of Method. You can use Kernel#method or Kernel#singleton_method to get the object of method, then use Method#call to run the method. Also you may use Method#to_proc and define_method to convert method and proc to each other. Eventhough the method still has scope of it’s defined.

Unbound Methods

Method object can also be unbound from original scopes by using Method#unbind, or Module#instance_method. Generated UnboundMethod object can not be called directly, but you can still call it after bind it to another scope by using UnboundMethod#bind. The only notice is that UnboundMethod object can only be bound to same class or sub if it’s from class, and no such limitation when it’s from module.

Comments

Metaprogramming Ruby: Core Concepts - Open Classes & Refinements

Opening Classes

Opening classes brings much flexibility for ruby. It permits you to modify existed classes or modules permanently, without having to implement whole world in case you just need to give a simple patch for current tool. For example:

1
2
3
4
5
class String
  def to_alphanumeric
    gsub(/[^\w\s]/, '')
  end
end

to_alphanumeric is new interface to return only alpha and numeric part of a string. However, opening class may not work well, especially it’s commonly dangerous to opening a frequently used target which would cause unexpected error in code. Due to this reason someone calls opening classes monkey patch and just leaves far from this feature.

While whatever, monkey patch continues conciseness in ruby, sometimes you are able to save your life gracefully with such powerful tool, except the trap behind it.

Refinements

From ruby 2.0, there is a more advanced form of monkey patch which is called refinements. Refinements are providing a way to extend classes only under specific scope, but not including modules. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
class C
  def foo
    puts "C#foo"
  end
end

module M
  refine C do
    def foo
      puts "C#foo in M"
    end
  end
end

Module#refine creates an anonymous module that contains the changes or refinements to the class (C in the example). self in the refine block is this anonymous module similar to Module#module_eval.

Scope

We use using to activate refinements defined in any places, but you may only activate refinements at top-level, not inside any class, module or method scope. You may activate refinements in a string passed to Kernel#eval that is evaluated at top-level. Refinements are active until the end of the file or the end of the eval string, respectively.

Refinements are lexical in scope. When control is transferred outside the scope the refinement is deactivated. This means that when you require or load a file or call a method that is defined outside the current scope the refinement will be deactivated.

If a method is defined in a scope where a refinement is active the refinement will be active when the method is called.

When defining multiple refinements in the same module, inside a refine block all refinements from the same module are active when a refined method is called.

You may also activate refinements in a class or module definition, in which case the refinements are activated from the point where using is called to the end of the class or module definition.

Method lookup under refinement

When looking up a method for an instance of class C Ruby checks:

1. If refinements are active for C, in the reverse order they were activated:

The prepended modules from the refinement for C

The refinement for C

The included modules from the refinement for C

2. The prepended modules of C
3. C
4. The included modules of C

If no method was found at any point this repeats with the superclass of C.

Note that methods in a subclass have priority over refinements in a superclass. For example, if the method / is defined in a refinement for Integer 1 / 2 invokes the original Fixnum#/ because Fixnum is a subclass of Integer and is searched before the refinements for the superclass Integer.

If a method foo is defined on Integer in a refinement, 1.foo invokes that method since foo does not exist on Fixnum.

Super

When super is invoked method lookup checks:

The included modules of the current class. Note that the current class may be a refinement.
If the current class is a refinement, the method lookup proceeds as in the Method Lookup section above.
If the current class has a direct superclass, the method proceeds as in the Method Lookup section above using the superclass.

Note that super in a method of a refinement invokes the method in the refined class even if there is another refinement which has been activated in the same context.

Refinements and module inclusion

Refinements are inherited by module inclusion. That is, using activates all refinements in the ancestors of the specified module. Refinements in a descendant have priority over refinements in an ancestor.

After all, refinement is still an experiment feature in ruby 2.x series, there is a detailed specification in official website, which should cover most aspect of it.

Comments

Metaprogramming Ruby: Core Concepts - Scope & Self

Scope defines the circumstance around any ruby statement, which basically limits your accessibility to resources. But ruby also has implicit and complicate Constants/Methods lookup strategy to make things not so easy at all. This article will clarify the concepts about scope and self firstly, then deep into the Constants/Methods lookup strategy. Finally, let’s try to understand top-level which may be the most special case in this topic.

1. Scope

In most cases, the ‘Scope’ we say in ruby commonly refers to variable scope, which has a little bit different from other popular languages. For example, there are four basic variable scopes including local, global, instance and class level, none of them has business with each other. We’ve already know the naming specification below:

Name begins with Variable scope
[a-z] or _ local variable
$ global variable
@ instance variable
@@ class variable

Actually the tricky is not naming, it’s about default scope changing. As we know that ruby has lexical scope, but not like others, it does not support nested scope, like inner scope shares outer scope in some languages. Each of four variable scopes in above figure is purely independent to each other.

message & scope resolution operator

Dot . is message operator in ruby, mainly used for message receiving, like method call. Double colon :: is scope resolution operator, commonly used for getting constant in specific scope, Like Foo::Bar, means searching ./foo/bar with relative path. If Foo is undefined in current scope, you should always use ::Foo::Bar to figure it out using absolute path.

scope changing

There are four basic cases which may create and change the scope lexically: Class definitions (class), Module definitions (module), Methods (def) and Blocks. Block scope is a little bit different from others and more interesting, which would create its own scope after the block defined, but also have closure capability, like code below:

1
2
3
4
5
foo = ''
bar = lambda { foo = 'hi' if foo == 'hello' }
foo = 'hello'
bar.call
foo #hi

For sure we can break this very easily:

1
2
3
4
5
foo = ''
bar = lambda { |;foo| foo = 'hi' if foo == 'hello' }
foo = 'hello'
bar.call
foo #hello

Notice that |;foo| defines foo as a local variable in its block, this kind of feature was introduced from ruby 1.9.

constants scope

The scope we’ve talked until now basically only means variable scope. In ruby, constant is totally different type of data compared with other languages like C++ or Java, any constant name must be started with uppercase character. And none of them can be assigned dynamically. Like:

1
2
3
def foo
  Bar = 'hello'
end #should give SyntaxError for dynamic constant assignment

Constants defined within a class or module can be accessed within its parent. And scope operator we talked before should be used when it’s outside the class or module. Constants defined outside any class or module are global scope, can be accessed directly or by using the scope operator ‘::’ with no prefix.

2. Self

As we know that the grail in ruby world would be conciseness. However, such conciseness is built on many bright solutions like keyword self.

if you just type a name in ruby console, it will show you an error:

1
foo #NameError: undefined local variable or method `foo' for main:Object

Indeed it has a huge gap between ruby and other languages in such case. But the expression with only name as a statement in ruby really means asking for local variable first, if found nothing then go on for searching method defined by current object. Here ‘main’ is a special object which represents the top-level, will have more discussion in the last part of this article.

calling methods

When you want to call a method, there should always be a receiver pointed to target object. Commonly that would be in this form:

1
some_object.some_method

If some_object is omitted, some_method will be called on the object where the current scope belongs to. self is one of the keywords referring to the current object. Note that private is an exception in this case, we will show an example here which has already been introduced in last several articles talking about private and public.

1
2
3
4
5
6
def foo
  'Hello'
end

foo #Hello
self.foo #NoMethodError: private method `foo' called for main:Object

The most usage of self would be defining a class method, or getting singleton class of current class.

3. Method lookup

The order of name lookup should be local variable/constant, then method. We has discussed about variable and constant scope in the first part. Here let’s have a look at method lookup.

When a message call happens, ruby will found its receiver firstly which should be an object. Since all methods are stored in classes and modules so method lookup walks these, not the objects themselves.

Here is the order of method lookup for the receiver’s class or module R:

i. The prepended modules of R in reverse order.

ii. For a matching method in R.

iii. The included modules of R in reverse order.

If R is a class with a superclass, this is repeated with R‘s superclass until a method is found. And once a match is found method lookup stops. If no match is found this repeats from the beginning, but looking for method_missing. The default method_missing is BasicObject#method_missing which raises a NameError when invoked.

4. Understanding top-level

May be you’re not interested in top-level mechanism as a rails developer (You have to use top-level one day even you’re only develop rails application), but you need to notice that ruby is also regarded as a powerful scripting language and already as a built-in tool in many popular OS distributions. We talk about top-level because it behaves different from any ruby official documents.

I’ve found a nice explanation about top-level here, but there are some newer update in latest ruby, we’ll go through this.

Most of us may know that top-level refers to main, which is an object of Object. You can prove it very easily like this:

1
2
self #main
self.class #Object

But how about define a top-level method?

1
2
3
4
5
6
7
def foo
  'hello'
end
foo #hello
method(:foo).owner #Object
Object.private_method_defined?(:foo) #true
self.class.private_instance_methods(false) #[:foo]

Does that look like a class definition? Even for the constant definition:

1
2
Foo = 'hello'
Object.const_defined?(:Foo, false) #true

It just looks like you’re operating on class Object. Not only for this, we found that there can also use public, private, or include method, they are shadow in singleton_class of main:

1
singleton_class.private_instance_methods(false) #[:public, :private, :include, :using, :define_method]

Based on our analyst above, we find a strange dual-nature of ruby top-level with some unexpected definitions in main object in purpose. Matz gives a explanation about design of top-level. You can also get more discussion here. From my view, such unusual design exactly helps conciseness of this language, but also gains our confusion and curiosity.

Comments

Metaprogramming Ruby: Core Concepts - Object Oriented Hierarchies

Since last several articles, we’ve gone through all basic but the most important internal definitions in Ruby lang. They are BasicObject, Object, Module, and Kernel, don’t forget Class. The five composite the root of Object Oriented Hierarchies in Ruby world. One of their most useful benefit is incredible metaprogramming ability, which already provides people revolutionary tools like Ruby on Rails. Frankly speaking, the word metaprogramming makes itself too mysterious to be accepted by most people. Conciseness and Expressiveness - are the only purpose of this feature.

1. Class

Classes in Ruby are first-class objects, each class in Ruby is an instance of class Class. Typically, you create a new class by using: Class.new. When a new class is created, an object of type Class is initialized and assigned to a global constant (Name in this case). When Name.new is called to create a new object, the new method in Class is run by default. Classes, modules, and objects are interrelated, we will discuss more in next chapter.

Public Class Methods

new

Creates a new anonymous (unnamed) class with the given superclass (or Object if no parameter is given). You can give a class a name by assigning the class object to a constant. If a block is given, it is passed the class object, and the block is evaluated in the context of this class using class_eval.

Public Instance Methods

allocate

Allocates space for a new object of class’s class and does not call initialize on the new instance. The returned object must be an instance of class.

new

Calls allocate to create a new object of class’s class, then invokes that object’s initialize method, passing it args. This is the method that ends up getting called whenever an object is constructed using .new.

superclass

Returns the superclass of class, or nil. Returns nil when the given class does not have a parent class.

Private Instance Methods

inherited

Callback invoked whenever a subclass of the current class is created.

2. Hierarchy of Five Kings

For more clearly, here we give a figure to cover all five items listed in preface. (We also list two guests in this figure to represent custom codes in the hierarchy)

More advance Ruby OO hierarchy

From the figure, we can see that class can be completely separated with objects under well design. Ruby uses this kind of hierarchy to implement its internal Object Oriented design. Here we would like to introduce the responsibilities on metaprogramming for each in above figure, and have deeper discussion on Module and Class, which are directly related to custom codes.

BasicObject

BasicObject is the root of all objects in Ruby through its subclass Object. The only class method in BasicObject is ‘new’, which is used to allocate memory to new object, and do initialization. For the public instance methods, mainly including object_id, send, and instance_eval/instance_exec to provide basic operations on any objects in Ruby. Also, BasicObject figures out the method_missing mechanism, which is very important for Ruby DSL definition, and singleton methods hook — which actually connects classes and objects in Ruby.

Object

If any class is defined by using keyword ‘class’ without other specific inheritance, it should be inherited from Object by default. Object offers the top useful Constants in Ruby world, like ARGV, ENV, RUBY_VERSION, STDIN/STDOUT/STDERR, etc. Also for public instance methods, it provides objects comparison, clone/dup, freezing, and respond_to? checking, etc.

In metaprogramming field, Object mainly focus on instance level usage, like getting class of object, extend, instance_of?, methods, retrieving singleton_class/singleton_methods of object. etc.

Kernel

Kernel is very special in the hierarchy, because it’s absolutely not class but an object. On the other hand, Kernel is mixed into class Object, making it become the earliest members in the whole hierarchy and also play very important role in the process.

Unlike Object, Constants in Kernel do not refer to some parameters of internal Ruby lang, they’re more like an utility or tool kit for subclasses, especially the reference to popular classes. Kernel also plays hard in Process operations(abort, fork, spawn), Source file loading(auto loading, require), Exception(catch, fail, raise, throw), String(chomp, chop, format, gets, gsub, sprintf, sub), Controlling(loop, sleep, rand, srand, trace/untrace), and IO(open, print, put, readlines, select).

Kernel also enhances the metaprogramming, like binding, block_given?, caller, eval, lambda, and proc.

3. Modules and Classes

Module

Basically, Modules serve two purposes in Ruby: namespacing and mix-in functionality.

A namespace can be used to organize code by package or functionality that separates common names from interference by other packages. Mix-in functionality allows sharing common methods across multiple classes or modules. Ruby comes with the Enumerable mix-in module which provides many enumeration methods based on the each method and Comparable allows comparison of objects based on the <=> comparison method.

A module is created using the module keyword.

Class

Every class is also a module, but unlike modules a class may not be mixed-in to another module (or class). Like a module, a class can be used as a namespace. A class also inherits methods and constants from its superclass. Use the class keyword to create a class. Any method defined on a class is callable from its subclass, the same is true for constants. You can override the functionality of a superclass method by redefining the method. If you wish to invoke the superclass functionality from a method use super.

If you do not supply a superclass your new class will inherit from Object. You may inherit from a different class using < followed by a class name.

When used without any arguments super uses the arguments given to the subclass method. To send no arguments to the superclass method use super(). To send specific arguments to the superclass method provide them manually like super(2). super may be called as many times as you like in the subclass method.

Common properties

Note that there are many similarities between modules and classes. Besides the ability to mix-in a module, the description of modules also applies to classes.

Reopening

Reopening classes is a very powerful feature of Ruby, but it is best to only reopen classes you own. Otherwise it may lead to naming conflicts or difficult to diagnose bugs.

Nesting

Modules may be nested.

Packaging

Many packages create a single outermost module (or class) to provide a namespace for their functionality. You may also define inner modules using :: provided the outer modules (or classes) are already defined.

Self

self refers to the object that defines the current scope. And it will change when entering a different method or when defining a new module.

Constants

Accessible constants are different depending on the module nesting (which syntax was used to define the module). if you use :: to define A::B without nesting it inside A a NameError exception will be raised because the nesting does not include A. If a constant is defined at the top-level you may preceded it with :: to reference it.

Methods

Class methods may be called directly(This is slightly confusing, but a method on a module is often called a “class method” instead of a “module method”. See also Module#module_function which can convert an instance method into a class method.). When a class method references a constant it uses the same rules as referencing it outside the method as the scope is the same. Instance methods defined in a module are only callable when included. These methods have access to the constants defined when they were included through the ancestors list.

Visibility

Ruby has three types of visibility. The default is public. A public method may be called from any other object.

The second visibility is protected. When calling a protected method the sender must be a subclass of the receiver or the receiver must be a subclass of the sender. Otherwise a NoMethodError will be raised. Protected visibility is most frequently used to define == and other comparison methods where the author does not wish to expose an object’s state to any caller and would like to restrict it only to inherited classes.

The third visibility is private. A private method may not be called with a receiver, not even self. If a private method is called with a receiver a NoMethodError will be raised.

Alias and Undef

You may also alias or undefine methods, but these operations are not restricted to modules or classes.

4. Singleton classes

The singleton class (also known as the metaclass or eigenclass) of an object is a class that holds methods for only that instance. You can access the singleton class of an object using class << object like this:

1
2
3
4
5
6
7
8
9
class Foo
  class << self
    # self is the singleton class here
  end
end

class << Foo
  # self is the singleton class here
end

This allows definition of methods and attributes on a class (or module) without needing to write def self.my_method.

Since you can open the singleton class of any object this means that this code block:

1
2
3
4
5
bar = Object.new

def bar.bar_method
  1 + 1
end

is equivalent to this code block:

1
2
3
4
5
6
7
bar = Object.new

class << bar
  def bar_method
    1 + 1
  end
end

Understanding singleton class in ruby is very important to investigate internal ruby, especially for its OO design. In next article, we’ll introduce the scope mechanism in ruby.

Comments

Metaprogramming Ruby: Core Concepts - Kernel

The Kernel is the object of Module, thus it has all instance methods from the parent. Also it’s available as instance methods in every Ruby object since the Kernel is included by class Object. Kernel has widespread usage including but not limit to internal Constant references, Processes operations, Loading Strategies, Exceptions, Controlling, IO, and Basic string operations. This article will focus on metaprogramming on instance level.

Kernel

All Constants and instance methods are public in Kernel, thus they all can be used in any objects as callee.

binding

Returns a Binding object, describing the variable and method bindings at the point of call. This object can be used when calling eval to execute the evaluated command in this environment.

eval

Evaluates the Ruby expression(s) in string. If the binding is given, which must be a Binding object, the evaluation is performed in its context. If the optional filename and lineno parameters are present, they will be used when reporting syntax errors.

block_given? / iterator?

Returns true if yield would execute a block in the current context. The iterator? form is mildly deprecated.

callcc{|cont| block}

Generates a Continuation object, which it passes to the associated block. You need to require ‘continuation’ before using this method. Performing a cont.call will cause the callcc to return (as will falling through the end of the block). The value returned by the callcc is the value of the block, or the value passed to cont.call. In general, Continuation object is used to be analogous to C setjmp/longjmp, or a thread. cont will return itself, and cont.call will jump to end of callcc block.

caller

Returns the current execution stack—an array containing strings in the form file:line or file:line: in ‘method’. Returns nil if start is greater than the size of current execution stack. Optionally you can pass a range, which will return an array containing the entries within the specified range.

caller_locations

Returns the current execution stack—an array containing backtrace location objects. Returns nil if start is greater than the size of current execution stack. Optionally you can pass a range, which will return an array containing the entries within the specified range.

eval

Evaluates the Ruby expression(s) in string. If binding is given, which must be a Binding object, the evaluation is performed in its context. If the optional filename and lineno parameters are present, they will be used when reporting syntax errors.

global_variables

Lookup global variables.

lambda

Equivalent to Proc.new, except the resulting Proc objects check the number of parameters passed when called.

proc

Equivalent to Proc.new

Comments