ML Final Project

  • 106061119 李泓儒、106061212 賴傳堯

Dataset Reading & Preprocessing

In [1]:
from music21 import *
import glob 
notes = []
pre = 0
cur = 0

for file in glob.glob("midi_songs/*.mid"):

    midi = converter.parse(file)
    notes_to_parse = None
    parts = instrument.partitionByInstrument(midi)
    if parts: # file has instrument parts
        notes_to_parse = parts.parts[0].recurse()
    else: # file has notes in a flat structure
        notes_to_parse = midi.flat.notes
    for element in notes_to_parse:
        if isinstance(element, note.Note):
            cur = float(element.offset)
            offset = round((cur - pre), 3)
            pre = cur
            notes.append([str(element.pitch), offset, element.quarterLength])
            
        elif isinstance(element, chord.Chord):
            cur = float(element.offset)
            offset = round((cur - pre), 3)
            pre = cur
            notes.append([('.'.join(str(n) for n in element.normalOrder)), offset, element.quarterLength])

music21: Certain music21 functions might need the optional package matplotlib;
                  if you run into errors, install it by following the instructions at
                  http://mit.edu/music21/doc/installing/installAdditional.html

Read Data

  • 我們的Dataset是MIDI檔，用迴圈把.mid結尾的檔案一次一次讀進來
  • glob.glob（pathname)
    • 返回所有匹配的文件路徑列表。參數pathname定義文件路徑匹配規則。

Data Parsing

  • 我們利用Music21 library內的類別/方法來幫助處理資料
    • converter.parse(file): 將file嘗試轉換為Stream
    • instrument.partitionByInstrument(midi): 把midi這個Stream依照個別樂器分成不同Part
      • 如果結果parts中沒有東西，表示檔案並無樂器的架構('flat structure')，直接取得內容；如果有，用midi.flat.notes來取得其中的物件
    • note.Note、chord.Chord: 判斷取得的物件屬於哪一種，如果是Note，直接以string形式append；如果是Chord，則將其中包含的Note依序以'.'為分隔符號結合為string，再做append
    • 除了考慮Note和Chord之外，我們還考慮每個音/和弦之間的間距，或者說是曲速，計算方式是將每個音/和弦的offset與前一個相減(cur-pre)得到間距，再append進notes中作為一個獨立的element
      • offset是每個音/和弦在樂曲中的"位置"，經過相減後得到的是"間隔"
In [ ]:
midi.show('midi')
  • A sample of our dataset
In [2]:
import numpy
from keras.utils import np_utils

sequence_length = 100

# get all pitch names
pitchnames = set(tuple(I) for I in notes)


# create a dictionary to map pitches to integers
note_to_int = dict((note, number) for number, note in enumerate(pitchnames))
network_input = []
network_output = []

# create input sequences and the corresponding outputs
for i in range(0, len(notes) - sequence_length, 1):
    sequence_in = notes[i:i + sequence_length]
    sequence_out = notes[i + sequence_length]
    network_input.append([note_to_int[tuple(char)] for char in sequence_in])
    network_output.append(note_to_int[tuple(sequence_out)])

n_patterns = len(network_input)
# reshape the input into a format compatible with LSTM layers
network_input = numpy.reshape(network_input, (n_patterns, sequence_length, 1))
# normalize input
n_vocab = len(note_to_int)
network_input = network_input / float(n_vocab)
network_output = np_utils.to_categorical(network_output)

Using TensorFlow backend.
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorflow\python\framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorflow\python\framework\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorflow\python\framework\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])

Generating Training Data

  • sequence_length: 以一次100個Note/Chord為一個input sequence
  • Data Lableing: categorical to numerical:
    1. 將notes裡的元素做成set(過濾重複)
    2. 用enumerate建立mapping dictionary
  • Generating Input Sequences: (for迴圈)
    • sequence_in: 從notes第i開始100個，sequence_out: 第101個
    • network_input: 每個元素是一個list，裡面是代表note的數字，network_output: 一個代表note的數字
  • n_patterns: network_input總共有幾個list，就有幾種pattern
  • np_utils.to_categorical: 轉換為one-hot的形式

Building Model & Training

In [4]:
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, LSTM
from keras.optimizers import RMSprop
In [5]:
from keras import optimizers

rms = RMSprop(lr=0.01, rho=0.9)

model = Sequential()
model.add(LSTM(
        256,
        input_shape=(network_input.shape[1], network_input.shape[2]),
        return_sequences=True))
model.add(Dropout(0.3))
model.add(LSTM(512, return_sequences=True)) # return sequence not matrix
model.add(Dropout(0.3))
model.add(LSTM(256))
model.add(Dense(256))
model.add(Dropout(0.3))
model.add(Dense(n_vocab))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer=rms)

WARNING:tensorflow:From C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\keras\backend\tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\keras\backend\tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\keras\backend\tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\keras\backend\tensorflow_backend.py:133: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.

WARNING:tensorflow:From C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\keras\backend\tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\keras\optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING:tensorflow:From C:\Users\USER\Anaconda3\envs\tensorflow_GPU\lib\site-packages\keras\backend\tensorflow_backend.py:3295: The name tf.log is deprecated. Please use tf.math.log instead.

RNN

  • Building RNN by adding layers using Sequential()
    • 第一層必須提供input_shape: network_input.shape[1], network_input.shape[2]
    • 512、256: 各層的 node數
    • 0.3: 要捨棄的比例
    • categorical cross entropy: 計算每個training epoch的loss(適合多個class)
    • RMSprop optimizer: 通常適合RNN
      • 用keras的RMSprop class做optimizer，調整learning，選擇lr=0.01，training的收斂速度比較快。
  • Layers
    • LSTM: RNN的主要layer
    • Dropout: 可以設定捨棄一定比例的weight，避免overfit
      • 選擇0.3，比較不會對得到的weight影響太大(捨去不會太多，也不會導致training時間太長、overfit)
    • Dense: fully connected的一層
    • Activation: 在這層會決定用甚麼activation function計算最後的output
In [6]:
# import tensorflow，用gpu跑，加快training速度
import tensorflow as tf
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))

from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.allocator_type = 'BFC' #A "Best-fit with coalescing" algorithm, simplified from a version of dlmalloc.
config.gpu_options.per_process_gpu_memory_fraction = 0.3
config.gpu_options.allow_growth = True
set_session(tf.Session(config=config))
In [ ]:
# 測試是否是使用gpu
tf.test.is_gpu_available(
    cuda_only=False,
    min_cuda_compute_capability=None
)
In [ ]:
from keras.callbacks import ModelCheckpoint

filepath = "LSTM-weight-{epoch:02d}-{loss:.4f}.hdf5"    
checkpoint = ModelCheckpoint(
    filepath, monitor='loss', 
    verbose=0,        
    save_best_only=True,        
    mode='min'
)

callbacks_list = [checkpoint]
model.fit(network_input, network_output, epochs=200, batch_size=64, callbacks=callbacks_list)

Training

  • ModelCheckpoint reference
    • Save the model after every epoch.
    • filepath: can contain named formatting options, which will be filled with the values of epoch and keys in logs
      • if filepath is weights.{epoch:02d}-{val_loss:.2f}.hdf5, then the model checkpoints will be saved with the epoch number and the validation loss in the filename
    • monitor: quantity to monitor
    • verbose: verbosity mode, 0 or 1
    • save_best_only: if =True, the latest best model according to the quantity monitored will not be overwritten
    • mode: one of {auto, min, max}.
      • save_best_only=True, mode=min, the decision to overwrite is made based on the minimization of the monitored quantity.
  • model.fit() reference
    • train the network for 200 epochs (iterations)
    • each batch propagated through the network contains 64 samples

Generate Music

In [9]:
model = Sequential()
model.add(LSTM(
        256,
        input_shape=(network_input.shape[1], network_input.shape[2]),
        return_sequences=True))
model.add(Dropout(0.3))
model.add(LSTM(512, return_sequences=True)) # return sequence not matrix
model.add(Dropout(0.3))
model.add(LSTM(256))
model.add(Dense(256))
model.add(Dropout(0.3))
model.add(Dense(n_vocab))
model.add(Activation('softmax'))
# Load the weights to each node
model.load_weights('LSTM-weight-154-1.8321.hdf5')

Obtain Trained Model

  • 產生音樂使用的model必須要與training的model一模一樣
    • 所有的layer、參數都與前面一模一樣
    • 用model.load_weights把前面checkpoint存下的weight load進來
      • 選擇loss比較小的一組weight
In [40]:
start = numpy.random.randint(0, len(network_input)-1) # randomize starting point
int_to_note = dict((number, note) for number, note in enumerate(pitchnames)) # decoding(reversing) map
pattern = network_input[start] # a list of notes/chords
print(start)
print(len(pattern))
# create note and chord objects based on the values generated by the model
prediction_output = []
# generate 500 notes
for note_index in range(500):
    prediction_input = numpy.reshape(pattern, (1, len(pattern), 1))
    #print(prediction_input)
    prediction = model.predict(prediction_input, verbose=0)
    index = numpy.argmax(prediction) # the most likely one
    result = int_to_note[index] # map from int to note, then store as result
    print(result)
    prediction_output.append(result)
    pattern = numpy.append(pattern,[index/float(n_vocab)])
    pattern = pattern[1:len(pattern)] # update pattern

25828
100
('E-3', 1.0, 2.0)
('E5', 1.0, 3.0)
('A3', 0.5, 0.5)
('4.10', 0.5, 1.0)
('2.6', 0.0, 1.5)
('D5', 0.5, 2.5)
('0.3.5', 0.5, 1.0)
('A4', 0.25, 0.0)
('0.3', 0.0, 4.0)
('8.11', 0.0, 0.5)
('C#4', 0.5, 1.0)
('A5', 1.0, 1.0)
('2.5', 0.5, 1.5)
('5.9', 1.0, Fraction(1, 3))
('0.3.5', 0.5, 1.0)
('B-6', 0.25, 0.25)
('E5', 0.333, 2.0)
('E-5', 2.0, 1.0)
('B5', 0.25, Fraction(1, 3))
('8.0', 0.5, 0.5)
('G#5', 0.25, Fraction(1, 3))
('5.9', 1.0, 0.75)
('E-5', 0.333, 2.0)
('1.7', 1.0, 0.5)
('A5', 0.167, 1.0)
('E-4', 0.25, 0.75)
('B-6', 0.25, 0.25)
('7.10.1.3', 0.0, 0.25)
('D2', 1.0, 1.0)
('3.7', 1.0, Fraction(1, 3))
('G#4', 0.5, 0.5)
('D3', 0.25, 1.0)
('B-5', 0.25, 0.25)
('B-5', 2.0, 0.5)
('B-5', 0.25, 0.5)
('E5', 0.333, 0.0)
('B-3', 0.0, 0.25)
('3.9', 0.0, 1.0)
('B-5', 0.25, 0.25)
('B-5', 0.25, 0.5)
('G#3', 0.083, Fraction(1, 3))
('4.10', 0.5, 1.0)
('E5', 0.083, 0.0)
('2.5', 0.5, 1.0)
('5.8', 0.0, 4.0)
('2.5', 0.25, 1.0)
('E4', 0.25, 0.0)
('1.4', 0.25, 0.25)
('B-4', 1.5, 0.25)
('5', 0.0, 1.0)
('1.4', 1.0, 5.0)
('A5', 0.083, 0.0)
('1.7', 1.0, 0.5)
('B-2', 0.0, 3.0)
('F2', 0.0, Fraction(1, 3))
('A5', 0.167, 1.0)
('4.10', 0.5, 1.0)
('C#7', 0.25, 0.0)
('5.10', 0.0, 3.0)
('G#3', 0.083, Fraction(1, 3))
('A5', 0.167, 1.0)
('11.2.5', 1.0, 1.0)
('2.5', 0.25, 1.25)
('B-5', 0.0, 0.0)
('3.9', 0.0, 1.0)
('8.11', 0.25, 0.5)
('1.7', 1.0, 0.5)
('11.2.5', 1.0, 1.0)
('5.10', 0.0, 3.0)
('5.8', 0.0, 4.0)
('B-6', 0.25, 0.25)
('11.2.5', 1.0, 1.0)
('11', 0.5, 1.5)
('5.8', 0.0, 4.0)
('B-3', 0.25, 1.25)
('8', 0.0, 0.25)
('G#3', 0.083, Fraction(1, 3))
('10.2.5', 0.25, 1.0)
('G#5', 0.25, Fraction(1, 3))
('A5', 1.0, 1.0)
('9.0', 0.0, 1.0)
('G#3', 0.333, 0.25)
('E4', 0.25, 0.5)
('G#5', 0.25, Fraction(1, 3))
('8.0', 0.25, 1.0)
('8.0', 0.25, 1.0)
('B-3', 0.25, 1.0)
('A5', 3.0, 1.25)
('F#5', 0.0, 0.0)
('5.9', 1.0, Fraction(1, 3))
('1.4', 0.0, 0.75)
('B-3', 0.0, 0.25)
('2.6', 0.0, 1.5)
('9', 0.25, 0.0)
('F#5', 1.0, 2.0)
('B-4', 0.5, 0.25)
('D3', 0.0, 1.0)
('8.0', 0.25, 0.5)
('B5', 0.333, 0.0)
('E4', 1.25, 0.25)
('3.9', 0.0, 1.0)
('C5', 0.0, 1.5)
('3.9', 0.0, 1.0)
('G#5', 0.167, 0.5)
('B5', 0.333, 0.0)
('0.4', 0.0, Fraction(1, 3))
('8.0', 0.5, 0.5)
('0.4', 0.0, Fraction(1, 3))
('G#5', 0.167, 0.5)
('G4', 0.5, 2.0)
('0.4', 0.0, Fraction(1, 3))
('G4', 0.5, 2.0)
('0.4', 0.0, Fraction(1, 3))
('1.4', 0.0, 0.5)
('E4', 0.0, 1.0)
('G#3', 0.333, 0.25)
('F3', 0.25, 0.5)
('B-6', 0.25, 0.25)
('G#4', 0.25, 0.75)
('11.2.6', 0.0, 0.5)
('C5', 0.0, 0.25)
('5.9.0', 0.0, 5.0)
('G#4', 0.5, 0.5)
('F5', 1.0, 2.0)
('5.9.0', 0.0, 5.0)
('5.9.0', 0.0, 0.25)
('C5', 0.0, 0.25)
('C#5', 0.25, 0.0)
('2.6.9', 0.0, 0.25)
('5.9.0', 0.0, 0.25)
('E-3', 0.0, 0.5)
('F#4', 0.25, 2.0)
('9.0', 0.0, 2.0)
('F#4', 0.0, 3.0)
('7.10.1.3', 0.0, 0.25)
('5.9', 1.0, 0.75)
('C5', 0.0, 0.25)
('A5', 0.5, 1.5)
('B-4', 0.0, 0.25)
('5.7.10', 0.0, 1.0)
('C#6', 0.5, 1.0)
('2.5', 0.25, 1.25)
('C5', 0.0, 0.25)
('B-5', 0.0, 0.0)
('B-2', 0.0, 3.0)
('5.9', 1.0, 2.0)
('F5', 1.0, 2.0)
('3.7', 1.0, Fraction(1, 3))
('E-3', 0.0, 0.5)
('C#4', 0.0, 1.5)
('3.6', 0.0, 0.25)
('8.11', 0.0, 0.5)
('F6', 0.5, 0.0)
('3.8', 0.5, 0.25)
('F6', 0.5, 0.0)
('F6', 0.25, 0.5)
('7.11', 0.25, 1.5)
('5.8', 0.25, 4.0)
('F6', 0.25, 0.5)
('E4', 1.25, 0.25)
('F4', 1.0, 0.0)
('4.8', 0.25, 2.0)
('E5', 1.0, 3.0)
('8.0', 0.25, 1.0)
('B-4', 1.5, 0.25)
('4.10', 0.5, 1.0)
('E-3', 0.25, 0.25)
('G5', 1.0, 2.0)
('E-3', 1.0, 2.0)
('3.8', 0.0, 0.5)
('8.0', 0.25, 1.0)
('F6', 0.5, 0.0)
('F#4', 0.25, 2.0)
('F6', 0.25, 0.5)
('5.9', 1.0, 2.0)
('1.4.7', 1.0, 0.5)
('7.11', 0.5, 1.5)
('8.0', 0.25, 0.0)
('3.6', 0.25, 0.5)
('5.8', 0.25, 4.0)
('F6', 0.25, 0.5)
('11.2.5.7', 0.0, 4.0)
('3.6', 0.25, 0.5)
('C#6', 0.5, 1.0)
('C6', 0.167, 0.25)
('E-6', 0.333, Fraction(1, 3))
('2.4', 0.083, 1.0)
('11.2.5.7', 0.0, 4.0)
('5', 0.0, 1.0)
('2.4', 0.083, 1.0)
('C#3', 0.25, 0.0)
('F6', 0.333, Fraction(1, 3))
('E-3', 0.0, 0.5)
('D4', 0.167, 0.25)
('A5', 0.5, 1.5)
('11.2.5.7', 0.0, 4.0)
('8.11', 0.25, 2.0)
('A5', 0.333, 0.0)
('1.4.7', 1.0, 0.5)
('3.8', 0.5, 0.25)
('A5', 0.5, 1.5)
('E5', 0.083, 0.25)
('3.8', 0.5, 0.25)
('F6', 0.5, 0.0)
('3.6', 0.25, 0.5)
('3.8', 0.5, 0.25)
('D4', 0.167, Fraction(1, 3))
('E5', 0.083, 0.25)
('3.8', 0.5, 0.25)
('G#4', 0.5, 0.25)
('A5', 0.5, 1.5)
('3.6', 0.25, 0.5)
('F6', 0.5, 0.0)
('0.5', 0.25, 0.25)
('11.2.5.7', 0.0, 4.0)
('E4', 1.25, 0.25)
('1.4.7', 1.0, 0.5)
('11.2.5.7', 0.0, 4.0)
('B-5', 0.25, 0.0)
('7', 2.0, 0.25)
('11.2.5.7', 0.0, 4.0)
('5.8', 0.25, 4.0)
('5.8', 0.25, 3.0)
('11.2.5.7', 0.0, 4.0)
('E5', 0.333, 2.0)
('1.4.7', 1.0, 0.5)
('11.2.5.7', 0.0, 4.0)
('1.4', 0.0, 0.5)
('7.10.1.3', 0.0, 0.25)
('F6', 0.333, Fraction(1, 3))
('5.8', 0.25, 4.0)
('E-5', 2.0, 1.0)
('1.4.7', 1.0, 0.5)
('E5', 0.083, 0.25)
('9.0', 0.0, Fraction(1, 3))
('5', 0.0, 1.0)
('B-5', 0.25, 0.5)
('7.10.1.3', 0.0, 0.25)
('B3', 1.0, 0.25)
('5.8', 0.25, 3.0)
('D6', 0.167, 0.0)
('E5', 0.083, 0.0)
('C5', 0.0, 1.5)
('F#4', 0.25, 2.0)
('B-4', 1.5, 0.5)
('11.2.5.7', 0.0, 4.0)
('6.10.1', 0.5, 0.5)
('F4', 0.5, 0.5)
('B-5', 0.25, 0.5)
('8.11', 0.0, 0.5)
('8.0', 1.0, 1.0)
('3.7', 2.0, 0.5)
('F5', 1.0, 3.0)
('E-3', 0.0, 0.5)
('11.3', 1.0, 1.75)
('E-3', 1.0, 2.0)
('C#4', 0.5, 1.0)
('F6', 0.25, 0.5)
('F6', 0.5, 0.0)
('F4', 0.5, 0.5)
('B-4', 0.0, 0.5)
('8.0', 0.25, 0.0)
('F6', 0.25, 0.5)
('F#5', 1.0, 2.0)
('3.6', 0.25, 0.5)
('2', 0.5, 1.0)
('C#6', 0.5, 1.5)
('11.1', 0.5, 0.5)
('F4', 0.5, 0.5)
('3.6', 0.25, 0.5)
('3.6', 0.25, 0.5)
('3.6', 0.25, 0.5)
('C#6', 0.5, 1.5)
('3.6', 0.25, 0.5)
('3.6', 0.25, 0.25)
('11.1', 0.5, 0.5)
('3.6', 0.25, 0.5)
('3.6', 0.25, 0.5)
('D4', 0.167, Fraction(1, 3))
('3.6', 0.25, 0.5)
('8.10', 0.0, 0.5)
('F6', 0.25, 0.5)
('11.2.5.7', 0.0, 4.0)
('C4', 0.333, Fraction(1, 3))
('8.0', 0.25, 1.0)
('F6', 0.5, 0.0)
('D4', 0.167, Fraction(1, 3))
('11.2.5', 1.0, 1.0)
('E-3', 1.0, 2.0)
('F6', 0.5, 0.0)
('11.2.5.7', 0.0, 4.0)
('C4', 0.333, Fraction(1, 3))
('8.0', 0.25, 1.0)
('A5', 0.0, 0.5)
('8.10', 0.25, 0.5)
('E-5', 2.0, 1.0)
('9.0', 0.5, 0.25)
('D4', 0.167, 0.25)
('D4', 0.167, 0.25)
('G5', 0.167, 0.0)
('B-4', 1.5, 0.25)
('2.5', 0.0, 1.0)
('C#3', 0.25, 0.0)
('C#3', 0.25, 0.0)
('8.11', 0.25, 0.5)
('G#5', 0.333, Fraction(1, 3))
('G#5', 0.333, Fraction(1, 3))
('0.5', 0.0, 0.25)
('6', 0.0, 0.25)
('B2', 0.0, 3.0)
('C5', 1.5, 0.5)
('E5', 1.0, 3.0)
('B-3', 0.25, 1.0)
('2.4.8', 0.5, 1.5)
('0.3.5', 1.0, 0.25)
('B2', 0.0, 3.0)
('D5', 0.5, 2.5)
('6.9.11', 0.5, 1.5)
('0.3.5', 1.0, 0.25)
('A4', 0.25, 0.5)
('2.4.8', 0.5, 1.5)
('2.5', 0.5, 1.0)
('2.5', 0.0, 1.0)
('4.7.9', 1.5, 1.5)
('G2', 0.0, 2.0)
('2.4.8', 0.5, 1.5)
('5', 0.0, 1.0)
('A4', 0.25, 0.0)
('2.4.8', 0.5, 1.5)
('1.4', 0.0, 0.5)
('C5', 0.0, 1.5)
('G#5', 0.25, Fraction(1, 3))
('2.5', 0.5, 0.5)
('1.4.8', 0.333, 1.0)
('8.0', 1.0, 1.0)
('C4', -442.5, 0.5)
('F5', 0.5, 0.75)
('8.0', 0.5, 0.5)
('F6', 0.333, Fraction(1, 3))
('2.5', 0.5, Fraction(1, 3))
('9', 0.25, 0.0)
('F6', 0.25, 0.5)
('8.0', 1.0, 1.0)
('C4', -442.5, 0.5)
('3.8', 0.5, 0.25)
('3.8', 0.5, 0.25)
('F6', 0.5, 0.0)
('3.6', 0.25, 0.5)
('7.10', 0.5, 1.0)
('1.4.7', 1.0, 0.5)
('11.2.5.7', 0.0, 4.0)
('3.7', 0.5, Fraction(1, 3))
('B-4', 0.0, 0.25)
('2.4.8', 0.5, 1.5)
('11.2.5.7', 0.0, 4.0)
('11.2.5.7', 0.0, 4.0)
('11.2.5.7', 0.0, 4.0)
('11.2.5', 1.0, 1.0)
('D5', 0.5, 2.5)
('C#5', 0.25, 0.0)
('1.3', 1.0, 1.0)
('B5', 0.333, 0.0)
('11.2.5', 1.0, 1.0)
('A5', 0.5, 1.5)
('9.0', 0.0, Fraction(1, 3))
('B-4', 1.5, 0.5)
('3.8', 0.5, 0.25)
('D4', 0.167, Fraction(1, 3))
('11.1', 0.5, 0.5)
('3.8', 0.5, 0.25)
('5', 0.5, 1.0)
('B-3', 0.0, 2.0)
('F4', 0.5, 0.5)
('D5', 0.0, 2.0)
('6.8.0', 1.0, 2.5)
('F6', 0.25, 0.5)
('1.5.8', 0.25, 1.0)
('1.4.7', 1.0, 0.5)
('8.0.3', 0.75, 1.25)
('5.8', 0.25, 3.0)
('A5', 0.167, 1.0)
('11.2.5', 1.0, 1.0)
('A5', 0.5, 1.5)
('7.11', 0.25, 1.5)
('A5', 0.5, 1.5)
('G5', 0.167, 0.0)
('B-2', 0.0, 3.0)
('5.8', 0.25, 4.0)
('B-5', 0.25, 0.25)
('6.11', 0.5, 3.0)
('A3', 0.5, 0.5)
('A3', 0.5, 0.5)
('5.8', 0.25, 4.0)
('5.8', 0.25, 4.0)
('A3', 0.5, 0.5)
('5.9', 1.0, 0.75)
('A3', 0.5, 0.5)
('5.8', 0.25, 4.0)
('1.3', 1.0, 1.0)
('E5', 0.333, 2.0)
('5', 0.0, 1.0)
('5.9', 1.0, 2.0)
('A5', 0.5, 1.5)
('5.8', 0.25, 4.0)
('5.8', 0.25, 4.0)
('E-3', 0.0, 0.5)
('5', 0.0, 1.0)
('E-3', 0.0, 0.5)
('3.8', 0.5, 0.25)
('E-6', 0.333, Fraction(1, 3))
('E-3', 0.0, 0.5)
('3.8', 0.5, 0.25)
('B5', 0.333, 0.0)
('5.8', 0.25, 3.0)
('E5', 0.083, 0.25)
('G#1', 0.25, 0.5)
('B5', 0.333, 0.0)
('1.4.7', 1.0, 0.5)
('A4', 0.5, 4.5)
('B-4', 0.0, 0.25)
('8.10', 0.25, 0.5)
('B-3', 0.0, 2.0)
('B-5', 0.25, 0.75)
('B-4', 0.0, 0.25)
('11.1', 0.5, 0.5)
('D4', 0.167, 0.25)
('3.7', 2.0, 0.5)
('3.8', 0.5, 0.25)
('7', 2.0, 0.25)
('7', 2.0, 0.25)
('11.2.5', 1.0, 1.0)
('11.2.5.7', 0.0, 4.0)
('A5', 0.5, 1.5)
('8.0', 0.25, 1.0)
('2', 0.5, 1.0)
('1.4.7', 1.0, 0.5)
('E-6', 1.75, 0.25)
('1.4.7', 1.0, 0.5)
('G4', 0.5, 2.0)
('1.4.7', 1.0, 0.5)
('1.6', 1.0, 0.5)
('3.7', 2.0, 0.5)
('B-6', 0.25, 0.25)
('7', 2.0, 0.25)
('B5', 0.333, 0.0)
('B-5', 0.333, Fraction(1, 3))
('7.11', 0.25, 1.5)
('11.2.5', 1.0, 1.0)
('B-6', 0.25, 0.5)
('7', 2.0, 0.25)
('11.2.5.7', 0.0, 4.0)
('7.11', 0.5, 1.5)
('1.4', 1.0, 5.0)
('2.4.8', 0.5, 1.5)
('3.8', 0.5, 0.25)
('11.2.5.7', 0.0, 4.0)
('6.9', 0.333, 1.0)
('6.9', 0.333, 1.0)
('E5', 0.083, 0.25)
('11.1', 0.5, 0.5)
('7.11', 0.25, 1.5)
('3.6', 0.25, 0.5)
('3.6', 0.25, 0.5)
('F#5', 0.25, 1.5)
('1.4.7', 1.0, 0.5)
('3.6', 0.25, 0.5)
('G5', 0.167, 0.0)
('1.4.7', 1.0, 0.5)
('3.6', 0.25, 0.5)
('G5', 0.167, 0.0)
('6.9', 0.333, 1.0)
('2.4.8', 0.5, 1.5)
('3.8', 0.5, 0.25)
('B-5', 0.25, 0.5)
('B-4', 0.0, 0.25)
('8.0', 1.0, 1.0)
('G#4', 0.25, 0.75)
('5', 0.0, 1.0)
('E-5', 0.333, 0.5)
('8.0', 1.0, 1.0)
('2.4.8', 0.5, 1.5)
('B3', 1.0, 0.25)
('6.10.1', 0.0, 0.25)
('F4', 0.5, 0.5)
('F4', 0.5, 0.5)
('8.0.3', 0.75, 1.25)
('F4', 0.5, 0.5)
('3.6', 0.25, 0.25)
('9.0', 0.5, 0.25)
('2.4.8', 0.5, 1.5)
('3.6', 0.25, 0.25)
('F4', 0.5, 0.5)
('2.5', 0.25, 1.25)
('C5', 0.0, 1.5)
('2.4.8', 0.5, 1.5)
('3.6', 0.25, 0.25)
('B5', 0.25, Fraction(1, 3))
('E-5', 0.333, 0.5)
('C7', 0.167, 0.25)
('A5', 0.5, 1.5)

Predicting

  • numpy.random.randint(0, len(network_input)-1): 隨機選擇一個起始index
  • pattern = network_input[start]: 從network_input中取出起始sequence(會是一組一百個連續[音符、和弦, offset, duration]的list)
  • 進行500次預測，每次會得到單一個[音符、和弦, offset, duration]，並將預測結果append進下一次的input，擠掉input的第一個element，再接著下一次預測
  • 預測結果會由另外建立的dictionary做decode，從數字轉回原本的代表符號，append進prediction
In [34]:
prediction_output = [list(item) for item in prediction_output]
note_chord = [char[0] for char in prediction_output]
each_offset = [char[1] for char in prediction_output]
each_duration = [char[2] for char in prediction_output]
print(note_chord)
print(each_offset)
print(each_duration)

['A3', '5.8', '2.5.8', '11.2.6', 'G#5', 'C5', '5.8', 'B-5', '11.2.6', '5.9.0', 'C5', '3.7', 'C#5', 'B4', 'C5', 'G#4', 'B-3', 'C#4', 'G#5', 'B4', '8.10', 'E5', 'C#3', '7', 'C#3', 'E3', 'D6', '3.6', '5.9.0', '5.9.0', 'A5', 'A5', '6.11', 'G#1', 'G#5', '8.10', 'C#3', '5.9', '1.5', 'B-5', 'C5', '0.3.5', 'C#3', 'G#3', 'F#5', '7.9', 'A5', '6.11', '6.11', 'E-2', '8.0', '4.10', '3.6', 'E-3', 'C#3', '3.6.9', '5.10', 'B-4', '1', 'C#3', 'G4', 'G3', '11.2', 'G4', '0.4', 'E4', 'D3', 'B-5', 'B-3', '2.4', 'F5', 'G#4', 'B-6', 'E5', '3.6.9', 'A5', 'E4', '1.4', '7.9', 'F#4', '10.1', '10.1', '8', '8.10', 'B-2', '4.10', '5.8', 'E5', 'A5', 'E-3', '0.3.5', '3.8', '2.5.8', '5.8.0', '2.6.9', '8.10', 'C5', '5.8.0', 'E-7', 'E-6', '1', 'B-5', '5.8', 'E4', 'E4', 'F4', 'B-4', 'E-6', '7.10.1.3', '3.7', '2.5', '11', 'C5', '1.4.6', 'G#3', 'E-2', 'F#3', '8.10', '9', '4.10', '8.0', '5.10', '8.0', '11.2', 'C4', 'E4', '8.10', '8.0', 'C5', 'G4', 'F#5', 'G4', '0.3', 'D3', '5.9', 'B-5', '3.7', '7.10.1', '7', '2.4.8', 'F5', 'G5', '5.8', '5.8', '2.4.8', 'F5', 'F2', 'C4', '5.8', '5.8', 'B5', '1.4', 'B-5', 'F5', 'B-6', 'B-3', 'E4', '5.10', 'E4', 'C5', '2.5', '5.8', '1.5.8', 'B-6', 'C5', '1.4', 'F#5', '8', 'E-3', '3.7', 'E-3', 'B-4', 'G5', 'C#6', 'B-3', 'G5', 'E-6', 'B-3', 'F5', 'B-5', '7', '1.3', 'B-5', '0.3.5', 'C4', 'G4', '6.8', 'C6', '9', '2.6.9', 'A3', '7.11', 'A3', 'E5', 'F6', 'D4', '3.9', '0.3.7', 'C5', 'C5', 'E4', 'A3', 'F#3', 'B-5', 'C4', 'G#5', 'C#4', '2.5', 'B-3', 'C#4', 'F4', '5.8', 'G4', 'C5', 'E5', '3.7', 'G#3', 'B-4', 'C5', 'C3', '3.9', 'B-3', 'C5', 'G4', 'D3', '3.9', '8.0', 'E-5', '8.0', 'G3', 'G3', 'G4', '8.0', 'B-4', '0.4.7', '11.2.6', 'B-3', 'D5', '5.10', 'C4', 'D3', '0.5', '11', '6.9', '8.0', 'A5', '1.5.8', '2.5', '7.9', 'F#3', 'A5', '11.2.5', 'A5', 'D4', '7.9', 'C5', '8.0', 'F6', '3.6.9', '0.4.6', 'D4', 'E5', 'D4', '2.5', '2', '4.8', 'B-4', '11.2', '3.7', '4.10', 'E-6', 'C#5', 'F2', '2.5', 'B-3', '1.7', 'F6', 'B-6', '2.4.8', '8.10', '5.10', 'B-4', '3.8', 'A3', '2.4.8', '3.5', 'C5', '5.9.0', 'C5', 'E-6', 'C5', '3.7', 'B-6', 'C5', 'B-5', 'D4', 'F2', '8.0', '8.10', '3.8', '9.10', '6.10.1', 'G2', 'E4', '1.5.8', '5.9', 'G#4', '1.5', 'C6', 'C5', 'C#5', '4', 'F#3', 'B-5', 'E3', '10', 'G4', 'E-5', 'B-5', 'F6', '3', '8.0.3', 'B-5', '8.0.3', '1.4', 'A5', '9.10', '9.10', '3.7', '3.7', 'E5', '2.4.8', '3.8', '3.7', 'E-3', 'C#4', 'E5', 'E4', '11.2.5.7', 'F6', '7.9', '6.9', 'E-2', '1.4', 'F4', '4.7.11', 'A3', '1.7', 'B-6', 'D6', '0.3.5', 'B-6', '1.4.8', '5.8', 'D5', '8', 'C#5', '1.7', '1.7', 'D6', '7.0', '1.7', '1.4.8', 'A5', 'F6', 'D2', 'D2', 'D2', 'D2', 'D2', '9.0', 'C#7', 'F6', 'E3', 'B-2', 'D6', '1.7', '7', 'F#4', 'B2', 'E-3', '3.7', '3', '11.2.5', '5', 'C5', 'C#6', 'G#3', '1.4', 'B5', '7', '1.7', 'B5', 'B-5', 'E5', 'G5', 'F#4', 'A5', 'E-5', 'B-5', 'C4', '8', '1.7', 'B-5', 'E5', 'F#5', '3', '5.7.10', 'C#5', 'C5', 'C6', '1.4', '11.2.6', 'E-6', 'B2', 'E-6', 'C#4', '3.8', 'C#4', '3.9', 'A5', '9.0', 'E-5', '0.3.6', 'F6', 'E-5', '8.0', 'C5', 'A5', 'C5', 'E-3', 'C#6', '9.1.4', 'C#6', '3.7', 'F6', '5.9', 'F6', '1.5.8', '2.4.8', '2.5.8', '2.5.8', 'B-5', 'F#5', 'G3', '4', '2.5', '11.2.6', 'C5', 'B5', '11.2', '1.6', '9.10', '1.4', 'E5', '1.3', 'C#3', '5.9.0', '3', 'F6', '5.9.0', 'C6', 'B-4', 'E4', '3.8', '9', '11', 'F#5', 'B4', 'C#6', '3.6', '7.11', 'D3', '3.9', 'B-2', 'F6', '11.2.5', '7.11', '8.0', '1.4.7', '3.8', 'F6', '1.4.7', '8.0', '7.11', 'C6', '1.4.7', 'C6', 'C5', 'E4', '7.11', 'C6', 'B-2', '2.4.8', '1.4', 'B-5', 'B-5', 'F6', 'B-5', '1.3']
[0.25, 0.0, 1.0, 0.5, 0.25, 1.0, 0.0, 0.25, 0.5, 0.0, 0.5, 2.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.25, 0.25, 0.5, 0.25, 0.25, 2.0, 0.25, 0.5, 0.167, 0.25, 0.0, 0.0, 1.0, 1.0, 0.5, 0.0, 0.25, 0.5, 0.25, 0.5, 0.0, 0.25, 0.0, 0.5, 0.25, 0.25, 0.0, 0.25, 0.167, 0.5, 0.5, 0.0, 0.25, 0.5, 0.25, 0.5, 0.25, 1.0, 0.5, 0.25, 0.5, 0.25, 0.0, 0.0, 2.0, 0.5, 0.0, 1.25, 0.25, 0.25, 0.0, 0.083, 1.0, 0.5, 0.25, 1.0, 1.0, 0.5, 1.25, 0.0, 0.25, 1.25, 0.5, 0.5, 0.0, 0.0, 0.0, 0.5, 0.0, 0.083, 1.0, 0.25, 1.0, 0.0, 1.0, 0.5, 0.0, 0.5, 0.0, 0.0, 0.25, 1.0, 0.5, 0.25, 0.0, 0.333, 0.333, 1.5, 0.0, 1.0, 0.0, 0.25, 0.5, 0.5, 0.75, 2.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.5, 1.0, 0.0, 2.0, 1.0, 0.5, 0.5, 0.5, 2.0, 0.5, 1.5, 1.0, 1.5, 0.0, 0.0, 1.0, 0.25, 2.0, 1.0, 2.0, 0.5, 0.75, 0.0, 0.0, 0.0, 0.5, 0.75, 0.0, 0.5, 0.0, 0.0, 2.0, 0.25, 0.25, 0.5, 0.25, 0.0, 0.333, 0.0, 1.25, 1.0, 0.5, 0.0, 0.25, 0.25, 1.0, 0.25, 0.75, 0.0, 0.0, 2.0, 0.0, 0.25, 0.0, 0.083, 0.0, 0.0, 1.75, 0.0, 0.75, 0.25, 2.0, 1.0, 0.25, 0.5, 0.5, 0.333, 1.0, 0.167, 0.25, 0.25, 0.5, 0.5, 0.25, 0.083, 0.5, 0.167, 0.0, 0.333, 1.0, 1.0, 0.333, 0.25, 0.333, 0.25, 0.5, 0.167, 0.5, 0.25, 0.0, 0.5, 1.5, 0.25, 0.5, 1.0, 1.0, 1.0, 0.5, 0.25, 0.0, 2.0, 0.0, 0.0, 1.0, 0.167, 0.167, 0.0, 0.25, 2.0, 1.0, 0.0, 0.0, 0.0, 0.5, 1.5, 0.0, 0.0, 0.083, 0.25, 0.0, 0.083, 0.0, 0.0, 0.5, 0.333, 1.0, 0.167, 0.25, 0.5, 0.25, 0.333, 0.167, 1.0, 0.167, 0.167, 0.25, 1.0, 0.25, 0.333, 1.0, 1.0, 0.5, 1.0, 0.5, 0.5, 0.5, 0.25, 1.5, 0.5, 2.0, 0.5, 1.75, 0.25, 0.0, 0.25, 0.0, 1.0, 0.25, 0.25, 0.5, 0.25, 0.0, 0.0, 0.0, 0.5, 0.5, 0.0, 0.5, 0.0, 0.5, 0.0, 0.5, 0.5, 0.25, 0.0, 0.333, 0.167, 0.0, 1.0, 0.25, 0.5, 0.5, 0.5, 0.0, 0.5, 0.25, 1.0, 0.5, 0.5, 0.167, 1.0, 0.25, 0.0, 0.333, 0.25, 0.25, 0.5, 0.0, 0.333, 0.25, 0.25, 0.5, 0.75, 0.25, 0.75, 1.5, 3.0, 0.5, 0.5, 2.0, 2.0, 1.0, 0.5, 0.5, 1.0, 0.0, 0.0, 0.083, 0.333, 0.0, 0.25, 0.25, 0.333, 0.25, 0.25, 1.5, 0.0, 0.25, 1.0, 0.5, 0.167, 1.0, 0.25, 0.333, 0.0, 1.5, 2.0, 0.25, 1.0, 1.0, 0.167, 0.5, 1.0, 0.333, 3.0, 0.5, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.25, 0.25, 0.5, 0.0, 0.25, 1.0, 2.0, 0.25, 0.0, 0.0, 2.0, 0.25, 1.0, 0.0, 0.0, 0.5, 0.083, 0.25, 0.5, 2.0, 1.0, 0.25, 0.25, 0.333, 1.0, 1.25, 1.0, 0.333, 0.25, 0.5, 2.0, 1.0, 0.0, 0.25, 0.75, 0.5, 0.0, 2.0, 0.0, 0.25, 1.0, 0.5, 0.333, 0.0, 0.333, 0.333, 0.0, 0.333, 0.0, 0.5, 0.0, 2.0, 1.0, 0.25, 0.333, 0.25, 0.167, 0.5, 1.5, 0.0, 0.5, 0.0, 0.5, 2.0, 0.5, 1.0, 0.5, 0.25, 0.5, 1.0, 1.75, 0.25, 1.0, 1.0, 0.25, 0.5, 0.0, 0.5, 0.25, 1.0, 1.0, 0.5, 1.5, 0.083, 1.0, 0.25, 0.0, 0.5, 0.25, 0.0, 2.0, 1.5, 0.5, 0.5, 0.25, 0.0, 0.0, 0.0, 0.5, 0.25, 0.25, 0.167, 0.0, 0.0, 0.5, 1.0, 0.25, 1.0, 1.0, 0.0, 0.333, 1.0, 1.0, 0.25, 2.0, 1.0, 2.0, 0.0, 1.25, 0.25, 2.0, 0.0, 0.5, 0.0, 0.25, 0.333, 0.333, 0.25, 1.0]
[0.25, 4.0, 0.25, 0.25, Fraction(1, 3), 3.0, 4.0, 0.25, 0.25, 0.25, 0.0, 0.5, 2.0, 1.0, 2.0, 3.0, 0.25, 1.5, Fraction(1, 3), Fraction(2, 3), 0.5, 0.0, 0.0, 1.5, 0.0, 0.5, 0.25, 0.25, 5.0, 5.0, 1.0, 1.0, 3.0, 1.0, Fraction(1, 3), 0.5, 0.0, 0.5, 3.75, 0.5, 1.5, 0.25, 0.0, 1.0, 0.0, 1.0, 1.0, 3.0, 3.0, Fraction(1, 3), 1.0, 1.0, 0.25, Fraction(1, 3), 0.0, 1.0, 1.0, 1.25, 0.25, 0.0, 2.0, 0.0, 1.0, 2.0, Fraction(1, 3), 0.25, 1.0, 0.5, 0.25, 1.0, 3.0, 0.25, 0.5, 3.0, 1.0, 0.5, 0.25, 0.5, 1.0, 0.5, 0.25, 0.25, 0.25, 0.5, 3.0, 1.0, 4.0, 0.0, 1.0, 0.25, 0.25, 0.5, 0.25, 0.5, 0.5, 0.5, 1.5, 1.0, 1.0, 0.25, 0.5, 0.25, 4.0, 0.0, 0.0, 0.25, 0.25, 0.25, 0.25, 0.75, 1.0, 1.5, 0.25, 1.0, 0.5, Fraction(1, 3), 1.0, 0.5, 0.5, 1.0, 1.0, 3.0, 4.0, 2.0, 0.25, 1.5, 0.5, 1.0, 1.0, 0.25, 2.0, 0.25, 4.0, 1.0, 0.75, 0.25, 0.5, 3.0, 1.5, 1.5, 0.0, 0.5, 4.0, 4.0, 1.5, 0.0, Fraction(1, 3), 0.25, 4.0, 4.0, 1.0, 0.25, 0.25, 0.75, 0.5, 0.25, 0.0, 3.0, 0.25, 3.0, 1.5, 4.0, 1.0, 0.25, 3.0, 0.25, 0.25, 0.25, 0.5, 0.5, 0.5, 1.25, 0.5, 0.0, 0.25, 0.5, 0.25, 0.25, 0.0, 0.25, 1.5, 1.0, 0.25, 1.0, 0.25, Fraction(1, 3), 4.0, 0.25, 0.0, 1.0, 0.5, 1.5, 0.25, 0.0, 0.0, Fraction(1, 3), 1.0, 1.5, 3.0, 3.0, 0.0, 0.25, Fraction(1, 3), 0.25, 0.25, 0.5, 1.0, 1.25, 0.25, 1.0, 0.25, 4.0, 2.0, 3.0, 3.0, Fraction(1, 3), 0.25, 0.25, 1.5, 1.0, 1.0, 0.25, 3.0, 0.0, Fraction(1, 3), 0.25, 0.25, 1.0, 1.0, 0.0, 0.0, Fraction(1, 3), 0.5, 0.5, 1.5, 0.25, 0.0, 3.0, 3.0, 0.0, 1.0, 0.25, 1.5, 2.0, 1.0, 1.0, 1.0, Fraction(1, 3), 1.0, Fraction(1, 3), 1.0, 1.5, 1.0, 0.25, 1.0, 2.0, 1.0, Fraction(1, 3), 0.25, 2.0, 0.5, 3.0, 0.5, 0.25, 1.0, 2.0, 1.0, 0.25, 0.5, 1.0, 0.25, 0.0, Fraction(1, 3), 1.0, 2.0, 0.5, 0.5, 0.25, 1.5, 0.5, 3.0, 0.25, 0.5, 0.5, 1.5, 1.0, 1.0, 5.0, 1.0, 1.5, 1.0, Fraction(1, 3), 0.25, 1.5, Fraction(1, 3), 0.25, Fraction(1, 3), 1.0, 0.5, 0.25, 1.0, 0.5, 2.0, 1.5, 1.0, 2.0, 0.5, 0.5, 0.25, 3.0, 0.0, 1.0, Fraction(1, 3), 0.25, Fraction(1, 3), 1.0, 2.0, 2.0, 0.25, 0.5, 4.5, 1.25, 0.25, 1.25, 1.5, 1.25, 1.0, 1.0, 0.5, 0.5, 3.0, 1.5, 0.25, 1.5, 0.5, 1.5, 0.25, 0.0, 4.0, 0.5, 1.0, 1.0, 0.0, 0.25, 0.25, 0.25, 0.25, 0.5, 0.5, 0.25, 0.25, 0.5, 1.0, 4.0, 0.25, 2.0, 0.0, 0.5, 0.5, 0.25, 0.25, 0.5, 1.0, 1.25, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.5, 0.5, 3.0, 1.75, 0.5, 1.5, 2.0, 3.0, 0.5, 0.5, 1.5, 1.0, 1.0, 0.25, 0.25, Fraction(1, 3), 0.25, Fraction(1, 3), 1.5, 0.5, Fraction(1, 3), 0.25, 2.0, 0.5, 0.5, 1.0, 2.0, 0.25, 3.0, 2.0, 0.5, 0.0, 0.0, 0.25, 4.5, 1.0, 2.0, 1.5, 2.0, 5.0, 0.25, Fraction(1, 3), 3.0, Fraction(1, 3), Fraction(1, 3), 0.5, Fraction(1, 3), 0.25, 1.5, Fraction(1, 3), 1.0, 0.5, 0.5, 2.0, 1.0, 0.25, 1.5, 0.5, 0.5, 1.5, 0.75, 1.5, 0.5, 0.0, 2.0, 0.0, 1.0, 1.5, 0.25, 0.25, 0.25, 2.0, 0.25, 1.0, 1.5, 0.5, 0.0, Fraction(1, 3), 2.0, 0.5, 1.0, 1.5, 0.0, 1.0, 0.0, 0.25, 4.5, 0.0, 0.25, 0.5, 1.0, 1.5, 0.25, 0.0, 1.5, 0.0, 0.5, 1.0, 0.25, 1.5, Fraction(1, 3), 1.0, 3.0, 0.0, 1.0, 1.5, 1.0, 0.5, 0.5, Fraction(1, 3), 0.5, 1.0, 1.5, 0.5, 0.5, 0.5, 1.5, 0.25, 1.5, 0.5, 3.0, 1.5, 0.5, 0.0, Fraction(1, 3), Fraction(1, 3), 0.0, 1.0]

Manage result

  • 將結果分為三個list: Note和Chord、offset、duration，方便後面轉為midi檔
In [35]:
import random

offset = 0
output_notes = []

# create note and chord objects based on the values generated by the model
for pattern, off, duration in zip(note_chord, each_offset, each_duration):
    # pattern is a chord
    if ('.' in pattern) or pattern.isdigit():
        notes_in_chord = pattern.split('.')
        notes = []
        for current_note in notes_in_chord:
            new_note = note.Note(int(current_note))
            new_note.storedInstrument = instrument.Piano()
            notes.append(new_note)
        new_chord = chord.Chord(notes)
        new_chord.offset = offset
        new_chord.quarterLength = duration
        output_notes.append(new_chord)
    # pattern is a note
    else:
        new_note = note.Note(pattern)
        new_note.offset = offset
        new_note.storedInstrument = instrument.Piano()
        new_note.quarterLength = duration
        print(new_note.duration)
        output_notes.append(new_note)
    # increase offset each iteration so that notes do not stack
#    offset = offset + 0.5
    if (offset + off > 0) :
        offset = offset + off
    
midi_stream = stream.Stream(output_notes)
midi_stream.show("midi")
#midi_stream.write('midi', fp='test_output.mid')
start_pt.append(start)
midis.append(midi)

<music21.duration.Duration 0.25>
<music21.duration.Duration 1/3>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 2.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 2.0>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1/3>
<music21.duration.Duration 2/3>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.5>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.75>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 3.0>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 2.0>
<music21.duration.Duration 3.0>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.0>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 2.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.5>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1/3>
<music21.duration.Duration 2.0>
<music21.duration.Duration 1.5>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1/3>
<music21.duration.Duration 2.0>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.25>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.5>
<music21.duration.Duration 3.0>
<music21.duration.Duration 1.75>
<music21.duration.Duration 2.0>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.25>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 2.0>
<music21.duration.Duration 1.5>
<music21.duration.Duration 2.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 3.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 1.5>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.5>
<music21.duration.Duration 1.5>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 2.0>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1.5>
<music21.duration.Duration 0.0>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.5>
<music21.duration.Duration 0.5>
<music21.duration.Duration 1.5>
<music21.duration.Duration 0.25>
<music21.duration.Duration 0.5>
<music21.duration.Duration 3.0>
<music21.duration.Duration 0.0>
<music21.duration.Duration 1/3>
<music21.duration.Duration 1/3>
<music21.duration.Duration 0.0>

Result To MIDI File

  1. 每個迴圈利用predicted的結果string中是否有'.'符號來分辨是Chord還是Note
    • Chord需要將裡面的每個Note一一取出，append進新建一個notes的list
  2. 依照建立Chord或Note，並且設定各自的offset和duration，最後append進output_notes
    • 避免負的offset導致轉換成MIDI檔時出錯(原始檔中有可能計算出負的間隔，對原始檔來說是正常，但對預測結果來說，可能連續預測多個負值的間隔，使得總offset小於0)，因此多一層判斷，每次迴圈，offset只會加上正的間隔
In [38]:
midi_stream = stream.Stream(output_notes)
midi_stream.write('midi', fp='Final_Project.mid')
Out[38]:
'Final_Project.mid'
  • 匯出MIDI檔案

Summary

Idea Inspiration

  • How to Generate Music using a LSTM Neural Network in Keras
  • 在這篇文章中，他們使用了來自Final Fantasy的音樂做為dataset。透過機器學習(RNN)的方式，他們能夠自己產生一段音樂，有些聽起來真的很像電影編曲家寫的，引起我們的興趣，想要嘗試挑戰看看類似的題目。
    • 文章中並未考慮offset(他們固定間隔為0.5)，而在我們的報告中，納入了offset的考量，認為這樣的音樂會比較有趣、多樣化，然而這也使的training的時間變得很長，資料處理也變得比較複雜。

Data Preprocessing

  • Dataset:
    • 內容: 我們上網尋找適合的MIDI檔音樂，找到許多古時音樂家的鋼琴作品，我們決定挑選兩個古典時期代表音樂家:莫札特、貝多芬的音樂做為這次的dataset，好奇如果將這兩位音樂家的音樂混在一起，會產生出怎樣的旋律
    • 格式: MIDI檔，透過Music21 library方便處理
  • Input (Training) Data:
    • 必須將MIDI檔中的每個音符和弦挑出來，並取得offset、duration(方法前面都介紹過了)
    • 有一個值得討論的參數是，一次放多少個音符/和弦作為input(i.e. input sequence的長度)，我們是過25、50、100、200，其中25的training速度會快非常多，trade off就是結果比較容易出現重複，或者不和諧的音符，於是最後決定還是選100作為輸入長度 ### RNN Model
  • Layers:
    • 層數的安排以及順序主要是參考文章的內容建議的
    • drop rate稍微測試過，其實差別並不是很明顯，所以選擇保留原本的0.3
  • Training:
    • 一次train200個epoch，觀察每個loss的值，選擇loss相對較小的一組weight作為產生音樂的model weight
      • loss: 由預測出的音/和弦，與實際對應的output比較計算
    • 在不使用gpu的情況下train一次大概要花3~5天(一個epoch約30~60分鐘)，因此我們嘗試架設了tensorflow-gpu環境，用gpu執行，結果可以加快一次epoch大概6分鐘，但做完200個epoch大概還是要花一天，非常耗時，所以我們後來選擇較快收斂的learning(0.01)，觀察大蓋收斂後就會停止training ### Music Generation
  • W/out Considering Offset: 音樂聽起來中規中矩，聽起來比較舒適，但相對平淡(可以參考附檔: test_output.mid)
  • Considering Offset: 旋律比較多變，但有可能有間隔=0的時候(原始檔就有，間隔為0，但並未歸類為Chord)，這導致並不是一般一組Chord的音符重疊在一起，聽起來不大和諧，但加上速度(音語音之間的間隔、每個音的長度duration)，聽起來還是很有趣。 ### Conclusion
  • 這次的結果可以順利產生出許多不同的音樂(只要起始點不同，就會得到完全不同的結果)，且許多聽起來都不差，很有風味，雖然很難稱得上是古典時期的音樂，也比較難分辨是否有貝多芬/莫札特的風格，但某些部分還是可以感覺得出"可能真的有這首歌"的感覺。唯一可惜的是training太耗時間，還有許多的可能性因為時間上的限制無法更進一步改善或嘗試，所以存在一些不和諧的音階。不過換個角度解釋，也常有人刻意拿不和諧的音階作為悲傷或痛苦的詮釋，也可以為結果帶來一點不一樣的感覺。