Although multiple systems can be found throughout the Greek Magical Papyri and other writings from late antiquity, Claire Chandler provides a system of planetary and musical notation associations for the Greek vowels which Diane Touliatos worked out in her article “Nonsense Syllables in the Music of the Ancient Greek and Byzantine Traditions” published in The Journal of Musicology 7, no. 2 (1989) that I am partial to:

Alpha = Moon = D
Epsilon = Mercury = C
Eta = Venus = B flat
Iota = Sun = A
Omicron = Mars = G
Upsilon = Jupiter = F
Omega = Saturn = E

She also shares this explanation of how the vowels functioned in PGM XIII from Patricia Cox Miller’s “In Praise of Nonsense” from Classical Mediterranean Spirituality, ed. A. H. Armstrong (London: Routledge & Kegan Paul, 1986), p. 482:

When the God who is ‘an invisible symbol’ breaks into human speech, his sounds are the echoes of the alphabet, the vowels.

An example of which is the following:

Lord, I imitate you by saying the 7 vowels; enter and hear me!

A
EE
ĒĒĒ
IIII
OOOOO
YYYYYY
ŌŌŌŌŌŌŌ

ABRŌCH
BRAŌCH
CHRAMMAŌTH
PROARBATHŌ
IAŌ
OYAEĒIOYŌ

This could be fun to play around with, especially applied to “birdglyphic” and the enchanting language of falcons, as well as other operations from the PGM.