Copycat Agent

The Copycat musical agent serves as a gentle introduction to Euterpe. It's a simple agent that mimics the user's input. This straightforward interaction provides a good starting point for getting familiar with the Euterpe codebase and understanding how to work with the hook files.

Note

Before we start, rename the EmptyAgent folder in the /public/agents/ directory to Copycat. This will be the folder where we'll be working on our agent.

Configuration Files

General Configuration

• This agent operates on a note-based interaction, functioning as an event-based agent since the it only generates a note as an response to the user's input. Because of this we need to set the following flags in the config.yaml file:

  TIP

  We only provide the config flags that are relevant to the agent. You can leave the rest in their default state.

  config.yaml:

  title: "Copycat"
  # Set the correct flags based on the type of your music interaction algorithm
  interactionMode: 
    noteMode: true
    audioMode: false
  noteModeSettings:
    eventBased:
      # If eventBased.status: true, then the worker's processNoteEvent() hook 
      # is active for instant processing of each user's MIDI event.
      status: true
    gridBased:
      status: false

• Both the user and the agent use a piano instrument. There is no need for a metronome in this agent, so we don't include it as a player.

  config_players.yaml:

  players:
    human:
      label: 'User' # player labels must be unique
      mute: false
      volume: 7
      instruments: 
      # it must be one of the available instrument ids in audio/samples
      # you can't have two instruments with the same id in the same player 
      - id: "piano" 
        label: "Piano" # can be anything
        mute: false
        volume: 7
        default: true
    agent:
      label: 'Copycat'
      mute: false
      volume: 7
      instruments:
      - id: "piano"
        label: "Piano"
        mute: false
        volume: 7
        default: true

UI Configuration

Finally, in terms of UI widgets, we want to have PianoRoll and the on-screen Keyboard.We'll also use the TextBox to display the last note played by the agent. The score can't currently be used as it's tailored for grid-based and monophonic interaction modes.

In order to make it more interesting, we can choose to create 3 sliders that'll controll various aspects of the interaction.

• Randomness: This slider will control the randomness of the copycat. With a slider value of 0, the agent mimics the user precisely, but as the randomness value increases, the mimicry becomes less precise.
• Delay: This slider will control the delay of the copycat's response. With a slider value of 0, the agent mimics the user instantly, but as the delay value increases, the mimicry becomes more delayed.
• PitchShift: This slider will control the note distance between the user's input and the copycat's response. With a slider value of 0, the agent mimics the user 1 octave lower, but as the pitchShift value increases, the copycat's response becomes higher in pitch.

config_widgets.yaml:

gui:
  score: 
    status: false
  textBox:
    status: true
    title: "Note"
  pianoRoll:
    status: true
    human: true
    agent: true
  keyboard:
    status: true 
    # set the visible octaves. Max range [0,8]
    octaveStart: 2
    octaveEnd: 6
  chromaChart:
    status: false
  audioMeter: 
    status: false
  settingsModal:
    sliders:
      - id: 1
        label: "Randomness"
        value: 0
        min: 0
        max: 10
      - id : 2
        label: "Delay"
        value: 0
        min : 0
        max : 200
      - id : 3
        label: "Pitch Shift"
        value: 0
        min: 0
        max: 24

Note

The id for the each of the slider, needs to be unique within the sliders and in an ascending order starting from 1. Also each of these ids are used for mapping the slider's value to its corresponding agent-variable in the initAgent_hook.js file.

Hooks

initAgent_hook.js:

• Since the copycat's operation relies on a straightforward rule-based algorithm, we won't need to interact with the loadAlgorithm() and loadExternalFiles() hooks. These hooks are reserved for more complex agents requiring external resource loading.

  Within the initAgent_hook.js file, we can initialize our UI-related variables. There are two important points when defining a variable in the hooks:

  1. These variables must be defined as a property of the agent object. This ensures that the variable is accessible to all other hooks and not just the local scope of the initAgent_hook.js file. In order to do that we use the self keyword.

  self.randomness = 0.0; // CORRECT: variable accessible to all hooks
  let randomness = 0.0; // WRONG: variable only accessible to initAgent_hook.js

  2. In order to tie this variable to the slider we defined in the config_widgets.yaml file earlier, we need to add the following code in the switch statement of the updateParameter() function:

  case self.uiParameterType.SLIDER_1:
      self.randomness = newUpdate.value;
      break;

  NOTE

  Notice that the last token of the uiParameterType.SLIDER_1 is the same as the ID of the slider we created in config_widgets.yaml. This is why we need to make sure that the IDs are unique.

  To sum up, replace the variable definitions in the beginning of the initAgent_hook.js file and the updateParameter() function with the following code:

  // initAgent_hook.js
  self.randomness = 0.0;
  self.delay = 0.0;
  self.pitchShift = -12;
  
  export function updateParameter(newUpdate) {
      switch (newUpdate.index) {
      case self.uiParameterType.SLIDER_1:
          self.randomness = newUpdate.value;
          break;
      case self.uiParameterType.SLIDER_2:
          // Divide by 100 to make the delay range from 0 to 2 seconds
          self.delay = newUpdate.value/100;
          break;
      case self.uiParameterType.SLIDER_3:
          // Subtract 12 to make the pitch shift range from -12 to 12
          self.pitchShift = newUpdate.value - 12;
          break;
      default:
          console.warn('Invalid parameter type');
          break;
      }
  }

processNoteEvent_hook.js:

• The Copycat agent is a simple event-based agent, so we only need to implement the processNoteEvent() hook. This hook is invoked every time a new note is played by the user and receives a NoteEvent object as input argement. It then, constructs a new NoteEvent object, modifies it, and sends it back to Euterpe.

  Replace the processNoteEvent() function in the processNoteEvent_hook.js file with the following code:

  export function processNoteEvent(noteEvent) {
      // Euterpe expects a note list, even if it's a single note
      const noteList = [];
      // A dictionary to store the output to be sent to the UI
      const message = {};
  
      if (noteEvent.type == self.noteType.NOTE_ON) {
          // First we get the midi value of the note the user played
          const inputMidi = noteEvent.midi;
          // Then we estimate the midi the copycat will play
          let outputMidi = inputMidi + self.pitchShift + Math.floor(Math.random() * self.randomness);
          // Set the range of outputMidi to be 21-108 (piano range)
          outputMidi = clamp(outputMidi, 21, 108);
          // Set the text for the TextBox widget
          const label = outputMidi.toString();
          // and add it to the message to be sent to the UI
          message[self.messageType.LABEL] = label;
  
          // Construct a new NoteEvent object
          const copycatNote = new NoteEvent();
          // The player is the agent
          copycatNote.player = self.playerType.AGENT;
          // The instrument is required for playback
          copycatNote.instrument = self.instrumentType.PIANO;
          // The type of the note is the same as the user's input (note_on or note_off)
          copycatNote.type = noteEvent.type;
          // The midi value is the one we estimated
          copycatNote.midi = outputMidi;
          // The velocity is the same as the user's input
          copycatNote.velocity = noteEvent.velocity;
          // Play it with a delay determined by the slider
          copycatNote.playAfter = {
              tick: 0,
              seconds: self.delay,
          },
          // Push the note to the list of notes to be sent to the UI
          noteList.push(copycatNote);
  
          // We store the mapping of the user's note to the agent's note
          // in the userToAgentNoteMapping dictionary (defined in agent.js)
          // Later, when we receive the note-off event, we can use this mapping
          // to know which note to turn off
          self.userToAgentNoteMapping[noteEvent.midi] = [outputMidi];
      } else if (noteEvent.type == self.noteType.NOTE_OFF) {
          const midisToTurnOff = self.userToAgentNoteMapping[noteEvent.midi];
          for (const midiOff of midisToTurnOff) {
              // Construct a new NoteEvent object
              const copycatNote = new NoteEvent();
              copycatNote.player = self.playerType.AGENT;
              copycatNote.instrument = self.instrumentType.PIANO;
              copycatNote.type = noteEvent.type;
              copycatNote.midi = midiOff;
              copycatNote.velocity = noteEvent.velocity;
              copycatNote.playAfter = {
                  tick: 0,
                  seconds: self.delay,
              };
              noteList.push(copycatNote);
          }
      }
      /*
      At this stage, the worker has finished processing the note event.
      If there is some output that needs to be sent to the UI
      (e.g., a list of notes to be played), you can add it to a dictionary
      and return it. If not, it's fine to not return anything.
      */
      message[self.messageType.NOTE_LIST] = noteList;
      return message;
  }

Deploying the Agent

Now that we've implemented the Copycat agent, we can run it and see how it works. To do that, we need to switch to the Copycat agent in the Euterpe.vue file. Search for the agentName variable and set it to:

agentName: "Copycat",

• If you're using CodeSandbox, hit Ctrl+S (or Cmd+S) to save your changes, and then go to the tab that has the web app open and hit Ctrl+Shift+R (or Cmd+Shift+R) to do a hard refresh.

• If you're working locally on your laptop, start the deployment server (if it's not already running) by running the following command in your terminal:

  npm run dev

  If the deployment server was already running, make sure to do a hard refresh of the webpage (Ctrl+Shift+R) to see the changes.

Interacting with the Agent

• Press the 'Start' button (or hit the Spacebar) and play some notes. You should see the agent playing the same notes one octave lower.
• Open the settings modal (by clicking on the 'gear' button) and change the sliders to see how it affects the agent's mimicry.