Specify hyperparameter ranges for blocks

[haifeng-jin]

Feature Description

We want to enable the users to specify the value ranges for any argument in the blocks.
The following code example shows a typical use case.
The users can specify the number of units in a DenseBlock to be either 10 or 20.

Code Example

import autokeras as ak
from kerastuner.engine.hyperparameters import Choice

input_node = ak.ImageInput()
output_node = ak.DenseBlock(num_units=Choice("num_units", [10, 20]))(input_node)
output_node = ak.ClassificationHead()(output_node)
model = ak.AutoModel(input_node, output_node)

Note

Each pull request should only change one hyperparameter in one of the blocks.

Solution

Example pull requests are #1419 #1425 .
Here are the steps to follow:

1. You can just change any other argument in any other block supported by AutoKeras, as shown here.
2. Change the docstring. example
3. Make sure you imported the module. from kerastuner.engine import hyperparameters.
4. Change the typing of the argument. example
5. Change the saving mechanism to serialized objects. example
6. Change the loading mechanism to deserialized objects. example
7. Change how we initialize the hyperparameter to self. example Copy from where it is originally defined. example
8. Change how we use it. example

[lc0]

@haifeng-jin I would be happy to take this one

[haifeng-jin]

@lc0 Great! Thank you. Let me know if you have any questions.

[dineshkumarsarangapani]

@lc0 Are you planning to take all the blocks? Can you share something with me?

[lc0]

@dineshkumarsarangapani we can use this issue for coordination, so if you are taking any of the blocks, feel free to just mention it here

[dineshkumarsarangapani]

@lc0 Thank you. I will work on BertBlock and ConvBlock today and raise a PR in a day or 2.

[lc0]

@haifeng-jin we need to re-open this one again. Also how do you feel about adding a checklist of blocks to do?

[dineshkumarsarangapani]

@haifeng-jin we have to open this ticket. I think this is not done completely.

[mandalbiswadip]

I can take up Embedding, Transformer, and MultiHeadSelfAttention block in the coming few days.

[haifeng-jin]

@mandalbiswadip Thank you! please do. We don't have other people working on this right now.

[khayamgondal]

I can take XceptionBlock

[keremgocen]

looking for low hanging fruits

[Neproxx]

Hey, NickSmyr and I would like to contribute to this issue, so we are wondering whether it is still relevant and which blocks still need changes.

[haifeng-jin]

@Neproxx Great to see you like to contribute!
We don't have a complete list of which has been done and which to be done.
You may just work on any block that is not done yet.

[kutal10]

@haifeng-jin Just added a PR #1708 for one block

[NickSmyr]

Hey @haifeng-jin ,

Neproxx and I need to contribute to an open source project in the context of a university course (KTH DevOps course). We have proposed a PR a week ago (see #1710 ) for this issue and it would help us out a lot if you could review it and either merge it or tell us about necessary changes. If everything is fine, we would pose a PR for the rest of the hyperparameters that need to be changed. It is important to us to have this done within the next 10 days, so we would be very greatful if you could spare some time to look at the PR.

Many thanks!

[haifeng-jin]

@NickSmyr The PR is merged. Thanks for your contribution!
AutoKeras will have another release soon before TF 2.9.0 stable release.
Your commit will be in it.

[haifeng-jin]

@NickSmyr @Neproxx , if there is no other PRs from your side, I may start to draft another release.

[Neproxx]

@haifeng-jin yes, that was the last PR from our side.