Challenge Difficulty Levels
I have enjoyed reading Jonathan and Richard's posts summarizing a few of the FLL challenges. Specifically, I like that they are providing "Speculated Difficulty Levels". Here are some questions that might be fun for us to discuss:
What defines an easy challenge?
What factors combine to create a difficult challenge?
If you were designing the 2008 FLL Challenge, what types of missions would you add to challenge teams?
Comments
For example - scoring the non-competitive 15 points on the Satellite mission is very easy because: 1) It does not require accurate alignment 2) It is an easy straight shot from base 3) The manipulation required is at a medium height. Difficulty Rating 1/10
A harder mission would be retrieving the Uranium. It still is a straight shot, but then you'll have to rely on alignment. They are also semi close to a wall, so you'll have to make sure it doesn't interfere with your manipulator. Difficulty Rating 3/10
A hard mission is the Solar Panel. Precision is required. If there's a straight shot from base it's hard to get. Not gentle and you'll knock the panel off the house while trying to place it. Difficulty Rating 7/10
What would I have done to make it harder? How about for the corn and uranium missions, if you moved the red loop (as opposed to the neon-green or yellow) you lost all points for that mission. We'd see some really cool tools there.
Instead of bringing the rail car back to base by hand, how about with the robot (while fully out of base).
Maybe place the wind turbines on the sandy beach west of the house.
The sheer number of actions required this year will prove to be difficult for most. I think that we'll see some interesting solutions for the missions that require precision.
Josh
I think the two biggest factors that determine a challenge's difficulty are (1) the accuracy required by the robot in manipulating it successfuly, and (2) how complicated of a mechanism the robot needs to use to manipulate the model.
-Jonathan
Josh
>I think that the accuracy required
>and the complexity of the required
>manipulator are similar/the same.
Not necessarily... for example, the Buckyball/Bone mission last year required quite a bit of accuracy to get the ball right in the bone, but the mechanism could be as simple as something that dropped a ball - in fact you could have even made a simple tray that the ball would roll off of when the robot banged into the bone model. As long as the robot's accuracy was sufficient, the ball would roll right into the bone. :-)
Now take the Solar Panel on House mission this year. That requires about as much precision as the bone mission, but the mechanism for putting the Solar Panel on the roof has to be quite a bit more complex, since it has to lay it on an angle and let go of it gently (as well as putting it up high instead of dropping it).
Location is definitely an important factor, but I think of it as included in the accuracy factor. If the location is far away or in a tight spot, your robot will need more accuracy to manipulate it successfully.
-Jonathan
...I was very disappointed by the number of challenges in previous years that are of the form:
- build a complex mechanical arm
- point your robot in the right direction
- press the button
I'll decline to comment on whether the mechanical engineering aspects are more or less difficult than the programming aspects but I, for one, find them less interesting.
By contrast, I find the mechanically-simpler but programming-sophisticated robogames challenges far more compelling.
I'd like to challenges that need more use of sensors and logic than brute force. Perhaps with a requirement that you do all of the challenges without touching the robot between missions.
Kevin
>Not necessarily... for example, >the Buckyball/Bone mission last >year required quite a bit of >accuracy to get the ball right in >the bone
I disagree that this mission required much accuracy at all and would rate it's difficulty 2/10. Most teams approached the mission model from one of it's two sides. Getting there is easy and the side of the bone was large, making it hard to miss. Even with proper alignment, approaching the bone from the front made this a very simple mission.
>Now take the Solar Panel on House >mission this year. That requires >about as much precision as the >bone mission
You really think that the Solar Panel is the same difficulty as the Bone? The Bone had a large drop area compared to the size of the deliverable (maybe 10:1). The solar panel is 1.5-2:1 The smaller the ratio, the more difficult the mission because higher precision is required.
>but the mechanism for putting the >Solar Panel on the roof has to be >quite a bit more complex, since it >has to lay it on an angle and let >go of it gently (as well as >putting it up high instead of >dropping it).
I agree that the mechanism required is more complicated. This contributes to it being a more difficult mission.
>Location is definitely an >important factor, but I think of >it as included in the accuracy >factor. If the location is far >away or in a tight spot, your >robot will need more accuracy to >manipulate it successfully.
Location (as well as manipulation difficulty) attributes directly to the mission difficulty. The harder the mission, the harder to do it accurately. "Accuracy factor" = mission difficulty.
Linda - I agree that turns make missions difficult, but not enough to make the difficulty 5/10 (across the board). With proper programming you really shouldn't see more than a few percent difference. Turning 90 degrees vs 360 is significantly too much difference. Even 90 vs 180.
Josh
Wow, I haven't had that kind of experience at all, especially with the new NXT motors. Our turns were quite consistent, give or take a few degrees - nothing like the example you gave. Maybe the design you were using was causing the differences in turns?
Josh,
I'm not sure you're understanding me. I think that both the accuracy required in completing a challenge and the complexity required of the attachment influence the dificulty of a challenge. And I think one does not necessarily depend on the other. So for the Solar Panel mission, I certainly don't think it's as easy as the Bone mission, because it requires a more complex attachment, even though it has a similar required amount of accuracy.
Let me give you a simpler example of the difference between accuracy required and complexity of the attachment. Suppose the Oil Drill mission was in the different location you talked about (facing the west side with only 2" to spare). You could still use the same attachment you used when it was in the other position, but your robot would have to be more accurate in getting to the right position for the attachment to manipulate the model... see what I mean?
-Jonathan
IMO, In no way did the bone require anywhere NEAR the amount of accuracy the Solar Panel needs. Many teams approached the bone from one of the sides, allowing for a great amount of leeway. Most of the (few) failures I observed were from teams approaching from the end and not using well developed alignment devices, or using mechanisms to deposit the bucky ball that imparted too much speed to the ball during the delivery. I asked several teams that failed how long they had been in FLL and ALL were rookie teams.
Approaching the Solar Panel is quite a bit different. The robot will be placing the panel on the roof from a distance and so any degree of mis-alignment at the robot will translate into non-miniscule linear variations at the roofline.
Accordingly, if we could gather statistics, I'd be willing to bet that a MUCH higher percentage of teams successfully performed the bone mission than will perform the Solar Panel mission.