Introducing SingingGadgets – ScoreDraft Refactored

These days, I’ve been working on a new singing synthesis project, that is SingingGadgets:

https://github.com/fynv/SingingGadgets

We can call it a refactor of ScoreDraft, because the new project has a lot of functional overlapping with ScoreDraft.

The project SingingGadgets is started to address some architectural weakness of ScoreDraft when you see it as a Python library.

This article will explain briefly what are done in the refactor and why I did it, also giving an overview of the architecture of SingingGadgets.

More articles will be added to explain in detail about the Python interfaces that SingingGadgets provides.

Refactor: from ScoreDraft to SingingGadgets

There are 2 points of the refactor: PyPi Compatibility and Low-level Interface Exposing.

PyPi Compatibility

As a Python library, here we try to make it suitable to be distributed and deployed in a standard Python way, such as using PyPi.

There are more than one time that I’m asked: Could you pack and upload ScoreDraft to PyPi?

Frankly, I didn’t even know PyPi. I’ve been a C++ programmer for years, and ScoreDraft is actually my first Python project. The architectural design of ScoreDraft is not very much different from my other C++ projects, except I added it a Python shell for efficient input of musical notes, thanks to Python’s powerful interpreter. As a result, I found it not an option to upload ScoreDraft to PyPi when I did consider that seriously.

The picture above shows the architecture of ScoreDraft. As you can see, on the right side, starting from PyScoreDraft, this is clearly a plug-in based architecture, which is common in C++ applications. PyScoreDraft is a binary module (shared library) with Python interfaces, it dynamically loads the extensions (also shared libraries) at run-time. Note that these extensions are not Python extensions, but ordinary C++ shared libraries. Behind PyScoreDraft, there’s a typical C++ architecture, no Python at all. If you are familiar with Python’s Distutils/Setuptools, you will know that such architecture does not fit into Distutils’ building system.

What makes it worse is that ScoreDraft looks for voicebank data within its deployed directory. Again, this is common in C++ applications, but it sucks when you are expecting it to work as an Python library because no one wants to deploy voicebanks into Python’s library directory.

To make it suitable for standard Python library deployment, for SingingGadgets, I need to avoid C++ style dynamic loading of shared libraries, and make no assumptions about where the voicebanks are placed.

Low-level Interface Exposing

I’m also asked for more than one time: are you planning to write a GUI for ScoreDraft?

The answer is definitely NO. ScoreDraft has been designed to be used to enter musical notes directly as part of the script code. In other words, ScoreDraft is a complete application, and it is not designed as some building block of another system.

There’s indeed some module within ScoreDraft that can be called a singing synthesis engine, and can be used to build other systems, except that you have to do it in C++. There’s no Python interface directly exposed. Immediate behind ScoreDraft’s Python interface is the sequence parsing layer (in PyScoreDraft), and engines are behind that layer. Moreover, within the engine, UtauDraft, synthesis algorithms are much too coupled with the handling of UTAU voicebanks’ structure. I realized (only recently) that it should be better off to decouple the synthesis engine from a specific voicebank system, when you consider it from reusing’s aspect.

In SingingGadgets, I will expose a singing synthesis engine interface independent from any voicebank and input grammar directly to Python users. The sequence parsing layer, as well as voicebank handling code will all be rewritten in Python. Such design will maximize the possibility for Python programmers to use SingingGadgets together with other Python libraries to build their own customized systems, including those with a GUI.

Overview of the New Design

The SingingGadgets project is consisted of 2 packages: SingingGadgets and ScoreDraft.

Package SingingGadgets

The package SingingGadgets has the same name as this project. Indicating that it is the main library of the project. The package provides low-level interfaces to the core of singing synthesis. The most important module of the package SingingGadgets is called VoiceSampler, which is directly powered by a C++ module PyVoiceSampler. Another module TrackBuffer  is an utility for storing tens/hundreds of megabytes of waveform data with an external storage cache, also used for mixing. In the module UTAUUtils, there are several separated tools to handle UTAU voicebanks, such as reading OTO files, prefix-maps etc.

User can use these modules as gadgets as the name suggests. One example is shown by Test.py in the source-code repository.  In that case, there’s no limit about what kind of voicebank to use and how the music notes are acquired. Go ahead to build your GUI if you want.

After implemented singing synthesis, later, I also migrated some core functions of basic music synthesis to SingingGadgets, also with their low-level interfaces directly exposed, in BasicSamplers.py.

Package ScoreDraft

The package ScoreDraft has the same name as the project ScoreDraft, which is also the name of this site. This package can be seen as an example showing how to build an application layer atop SingingGadgets.  It can also be seen as special branch of the project ScoreDraft, or an alternative implementation of the previous project.

Like project ScoreDraft, package ScoreDraft defines a layer of custom grammar based on Python data containers (Lists/Tuples etc), allowing user to input musical drafts directly into Python source-code.

The module UtauDraft implements a singer interface backed up by VoiceSampler. The internal module Singer implements the sequence parsing logic, which is used by UtauDraft, and can be reused when there are other singer modules. The package also includes the utility scripts that are already written in Python in the project ScoreDraft, such as note definitions and lyric converters (not shown in the pic).

Basic musical synthesis interfaces InstrumentSampler and PercussionSampler are also reimplemented, which are based on the low-level interface exposed in BasicSamplers.py of package SingingGadgets.

The interface provided by package ScoreDraft is very similar to those provided by project ScoreDraft. An example of using the package ScoreDraft is given by TestScoreDraft.py in the source-code repository. However, be careful that there are indeed some differences between the interfaces of the package ScoreDraft here and the those provided in project ScoreDraft due to the rewriting.

Detailed usage explanations will come soon in the succeeding blogs.

Are We Deprecating the Project ScoreDraft?

No, we are not deprecating project ScoreDraft by starting this new project.

As stated, the “weakness” of ScoreDraft we are addressing here is only seen when we consider ScoreDraft as an Python library and try to use it as a building block to build another system (using Python).

The original intent of ScoreDraft was for it to be used as a complete singing and music synthesis application with Python as its input interface. If you consider it that way, ScoreDraft is still a good design which is very extendable. There are benefits of using C++ to organize modules, such as the efficiency of data sharing between modules. Therefore, the 2 systems will coexist in the foreseeable future.

About the Author

Leave a Reply

Your email address will not be published. Required fields are marked *