Aside from being handheld games with visual novel style these two series share a lot in common. Both of these series have completed two main trilogies. The first trilogy and the first game in the second trilogy in both of these series are drawn in 2D while the last two games in the series (the last 2 games in the second trilogy) are done in 3D on 3DS. The quality of these games is also waning and peaking at similar rates.
I decided to make a comparison list of the 6+6 main games in these series.
First Trilogy VS series
1. Phoenix Wright: Ace Attorney VS Professor Layton and the Curious Village
AA = +1 for von Karma, +1 for Edgeworth
PL = +1 for atmosphere and OST
-Both games are solid and never on the bottom of the game lists. The games introduce us to the cast and the settings of the game series and are in that sense quite astounding games for pulling in players over the years. I prefer AA1 for its story and PL for its gameplay. Since I prioritize story, Ace Attorney wins round 1.
2. Ace Attorney 2: Justice for All vs Professor Layton and the Pandora's Box
AA = +1 for von Karma
PL = +1 for the changing setting of the game. The train aspect is very enjoyable and you wait to get answers to questions laid right at the beginning.
-Both of these games are possibly the worst in their series. I used to hate both but after replaying, Pandora's box is superior due to being times quite enjoyable. Professor Layton wins round 2.
3. Ace Attorney 3: Trials and Tribulations vs Professor Layton and the Unwound Future
AA = +1 for story, +1 for Godot, +1 for plot twist, +1 for ending and last case
PL = +1 for story, +1 for antagonists, +1 for the setting, +1 for ending and the wrap-up of potlines.
-These last games of the first trilogy are often argued to be the best games in their respective series and it's kind of easy to see why. The stories in both of these games are intimate, they go to the past of our main characters and escalate from there to an emotional ending. The character writing in these games feels better. I used to prefer AA but after replaying I noticed some problems in the writing that take the game down. On the other hand I've grown to appreciate Unwound Future a lot more now. It's the best written PL game and the way it wraps up is great. It's less than 50/50 chance for Layton series to have an ending that doesn't disappoint but this game's way of wrapping up everything was just great. Professor Layton wins round 3.
Second Trilogy VS series
1. Ace Attorney 4: Apollo Justice vs Professor Layton and the Last Specter
AA = +1 for breaking the status quo a bit, +1 for story
PL = +1 for the story, +1 for presentation and ending
-This is 50-50 for me. I don't like Apollo Justice really while I enjoyed Last Specter. Last Specter is also presented better and it's enjoyable to find out how our main characters met. Both of these series also go to a bit more overarching elements in the series that are handled with later on. I think AJ might be better written if I looked at it with unbiased eyes but I just don't feel like replaying the game. I got to like 3rd-4th case on replay and had to stop, it bored me to tears. I feel like the writers changing back to Wright diminishes AJ more as well as it has this filler or side story taste to it. Professor Layton wins round 4.
2. Ace Attorney 5: Dual Destinies vs Professor Layton and the Miracle Mask
AA = +1 for story, +1 for plot twist, +1 for reintroduction
PL = +1 for story, +1 for splitting the game in past and present, +1 for presentation
-Both of these games suck in different ways. I think Dual Destinies is manufactured, very mechanical and plays it safe for the series. The game might be the most boring AA game as well to some due to being just average cases, but I recall it being competently written and the ending being acceptable. For Professor Layton the game is actually intriguing and tackles Layton's origins. It had potential to be a great game but it had an awful ending that gives a shit taste in my mouth. The presentation of the game is great, it has this atmosphere to it with the art and OST being used well. Because of the cringeworthy ending I just can't give it a win. Ace Attorney wins round 5.
3. Ace Attorney 6: Spirit of Justice vs Professor Layton and the Azran Legacy
AA = +1 for story, +1 for plot twist, +1 for setting and OST
PL = +1 for story, +1 for enjoyment, +1 for worldbuilding and characters
Ace Attorney OST are mostly computer generated trash but Spirit of Justice finally brought some real tunes to the series. The game started out unimaginably slow to introduce the player to Khura'in, a rural village with its messed up court system that you have to fight against. The game has some pretty meh fillers to it and it's longer than it should be but it is immersive and has a great ending and a lot of proper writing. It shows the evolving writing of the main game series.
Azran Legacy on the other hand might be the most enjoyable Layton game out of them all. Things immediately pick up unlike in the other games and you basically travel the world to many locations including old ones. I think all the settings in the game are great. The variety is a huge plus to me. Instead of a train and couple towns, there's just more special settings to the game. The storylines are also enjoyable as you learn about the world of the verse through dialogue, newspapers and adventuring. The game should have been longer though as it wrapped up in a flash. Its rushed ending diminished the impact since there was a lot it had to go through but it doesn't ruin the game (unlike Miracle Mask). Ace Attorney wins round 6.